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EES  SYSTEMS  LABORATORY,  CONTRACT  NO.F496208trK-0004^' 

AF08R  (FY82-84) 

ABSTRACT 


N«w  advanoai  in  understanding  human  naurooogni t i v«  functions  must  e 
from  new  technologies  for  antesuring  split-seoond  changes  in  event 
related  eleotrioel  end  magnet io  fields  whieh  may  soon  complement  the 
anatomioal  information  produoed  by  CT  and  NMR  brain  soans  and  the 
statio  metabol io  images  of  PET  soans* 


As  a  contribution  to  this  effort,  the  EE68L  is  developing  the  method 
of  Neurooogni  t  ive  Pmttmrn  INCP)  Analysis  to  study  both  sisiple  and 
oomptex  cognitive  tasks*  A  number  of  previous  findings  suggest  that 
neither  striotly  looal i sat ioni st  ('single  equivalent  dipole”)  nor 
equipotent ial i st  ('hologram')  amdeis  of  neurooogni t ive  prooessing  are 
realistio*  Rather,  a  more  appropriate  amdel  seems  to  be  a  distributed 
computational  network  in  whioh  there  is  continuous  oommunieat ion 
between  many  parallel,  specialised  prooessing  elements.  In  order  to 
better  awasure  and  model  these  oomplex  processes,  the  initial  thirty 
aMnths  of  this  projeot  fooused  on  developing  exper imentel  tasks  and 
advanoed  reoording  and  analysis  technologies.*^  The  following 
milestones  were  aohievedi  '  — ' 

1)  Development  of  a  6-seoond  bimodal  task  whioh  systemat ioal I y 
SMMiipulates  functional  relations  between  sensory,  motor  and 
association  oortioal  areas  by  varying  the  expeoted  and  actual  sensory 
smdality  of  maser io  stimuli  requiring  a  preoise  isometrio  finger 
response  and  resulting  in  feedbaok  about  response  aoouraoy*  This 
extends  the  observation  epooh  beyond  the  1-seoond  interval  most 
oosssonly  used  in  event  related  potential  (ERP)  studiesi 

2)  Implementation  of  a  system  for  on-line  reoording  and  monitoring  of 
up  to  S4  brain  potential  (BP)  ohannels,  for  improved  spatial  sampling, 
as  well  as  eye  movement  (EOS),  musole  potential  (EMS),  heart  rate  and 
respiration  signaisi 

9)  lap  I sawn tat  ion  of  a  ourrent  souroe  density  transform  to  convert 

scalp  eleotrio  field  distributions  into  smasures  of  ourrent  flow  into 
and  out  of  the  brain*  The  effeot  of  the  referenoe  eleotrode  is 
eliminated,  and  localisation  is  iaprovedi 

41  Expansion  of  the  A0IEE8  Analysis  system  to  a  64  channel  oapaoity* 
Impleswntat ion  of  an  analysis  exeoutive  for  autoatated  exeoution  of 
multi-stage  analyses  applied  to  900  megabyte  data  sets.  This  stakes 
detailed  analysis  of  an  unprecedented  aatount  of  data  possible! 

I 


> 
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51  DmlopMnt  of  improved  speotral  algorithm*  to  remove 
contamination  by  eye-movement  potential*  from  single-trial  BP  dataf 

•I  Development  of  improved  methods  of  averaged  evoked  time  series 
analysis,  inoludingi 

al  A  simple  algorithm  for  ERF  estimation  whioh  aohieves  a 
substantially  higher  signal -to-noise  ratio  without  the  imposition  of  t 
B£ iflfi  presumptions  about  the  oharaoter 1st ios  of  event-related  signal 
oomponentsi 

b)  The  first  application  of  Vigner  ( t ime/ frequency  domain! 
distributions  to  study  split-seoond  ohanges  in  the  frequenoy 
composition  of  evoked  t ime  series.  This  promises  to  be  a  useful 
rtatiroh  tool  whioh  provides  complementary  information  to  that 
obtained  from  traditional  t ime -dome in  maasuresi 

T)  Development  of  improved  methods  for  single-trial  pattern 
recognition  analysis,  inoludingi 

a)  A  rapid  single-ohannel  soanning  measure  to  seleot  trials, 
Intervals  and  ohannels  with  task-related  signals  for  further  analysis! 

bl  A  statistioal  ‘baseline*  estimation  procedure  whioh  provides  a 
nearly  neutral  reference  data  set  against  whioh  to  compare 
event-related  patterns.  This  eliminates  the  neoessity  of  using 
questionable  'resting  state*  or  prestimulus  baseline*! 

81  Trial  application  of  all  of  the  above  to  a  set  of  date  oonsisting 
of  about  500  trials  of  an  auditory  and  visual  peroeptuosmtor  task 
reoorded  with  48  BP  ohannels, 

Notei  I  terns  2-7  were  also  sponsored  by  the  Offioe  of  Naval  Researoh, 
the  Air  Foroe  Sohool  of  Aerospaoe  Medioine,  and  the  National  Soienoe 
Foundat ion. 
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D.  Summary  At  Research  Program  and  Results 

t*  Ovtrxiiw 

It  is  a  great  testimony  to  the  inventive  genius  of  the  first 
generation  of  psyohophysiologists  that  so  much  has  been  learned  from 
averaged  evoked  potentials  with  modest  recording  and  analysis 
technologies.  However,  ourrent  pessimism  expressed  by  many 
neuroso i ent i sts  about  future  gains  from  this  line  of  researoh  seems 
justified.  It  does  not  seem  likely  that  major  advances  in 
understanding  mass  neural  aotion  and  human  higher  brain  functions  will 
result  from  continued  application  of  essentially  twenty-year-old 
experimental  paradigms,  and  recording  and  analysis  technologies. 
Amazingly,  most  psyohophys iologi sts  are  oontent  to  study  variations  in 
one  or  two  averaged  evoked  potential  peaks,  and  dismiss  or  even  oppose 
the  development  of  more  advanoed  technologies.  It  is  obvious  that 
this  attitude  will  soon  lead  to  obsourity.  Fortunately,  there  is  a 
growing  impetus  for  developing  technologies  to  measure  split-seoond 


• '  •  *  «■  -  *  •  *  *  «.  *_  *  • ' 
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changes  in  functional  neuroanatomy  to  oompiement  the  anatomical  images 
generated  by  CT  and  NMR  brain  soans,  and  the  statio  metabolic  images 
of  PET  soans, 

Neurosoient i sts  who  are  accustomed  to  relating  the  activity  of  a  few 
neurons  to  aspeots  of  behavior  should  take  pause  to  consider  the 
future  of  their  own  line  of  researoh*  Simultaneous  recording  of  even 
a  few  dozen  neurons  will  not  reveal  the  emergent  properties  of 
funotional  neural  masses.  As  clearly  demonstrated  in  Walter  Freeman's 
latest  studies  (see  below),  appropriate  measurement  and  analysis  of 
field  potentials  reveals  information  not  otherwise  obtainable,  ie, 
two-dimensional  spat iotemporal  electrical  patterns  which  encode  the 
specific  properties  of  expected  and  perceived  odors.  This  amounts  to 
Ihl  demons t rat  ion  s±  the  existence  s±  MU  !£££  code" .  Whi le  there  are 
no  fundamental  obstaoles  to  extending  this  line  of  research  to  the 
neoeortex,  such  work  requires  use  of  advanced  recording  and  analytic 
technologies,  whioh  unf or tunate I y  are  foreign  to  most  neuroscientists, 

Basic  studies  using  animal  models,  particularly  pr imate  models,  are 
assent ial  in  studying  mass  neural  action  and  basic  perceptual,  motor 
and  oognitive  prooesses.  However,  an  animal  model  for  oomplex  human 
higher  brain  functions  is  not  possible.  If  we  are  to  study  such 
funotions,  with  the  goal  of  extending  the  limits  of  human  performance, 
it  is  neoessary  to  advance  the  state  of  the  art  of  methods  for 
studying  human  neurooogn i t i ve  funotioning,  This  will  require  an 
interdisoipl inary  effort  in  whioh  reoording  and  analytic  techniques 
with  increased  spatial  resolution  and  source  localization  capability 
are  applied  to  experimental  paradigms  more  closely  approximating 
normal  oogitive  functioning. 

As  a  contribution  to  this  effort,  the  EEQ  Systems  Laboratory  is 
developing  the  method  of  Neurooogn i t i ve  Pattern  (NCP)  Analysis  for 
quantifying  mass  neural  prooesses  related  to  perceptuomotor  and 
oognitive  aotivities.  At  the  same  time,  we  are  designing  experiments 
for  measuring  sequences  of  elementary  cognitions  while  recording  up  to 
64  brain  potential  (BP)  ohannels  and  several  other  physiological 
signals,  Qenerioally  speaking,  NCP  Analysis  currently  consists  of  the 
applioation  of  digital  s i gna I -prooess i ng  techniques  and  statistical 
pattern  classification  analysis  to  extract  small  task-related  signals 
from  the  obsouring  background  ‘noise*  of  the  brain  (Section  1.0,3). 

It  is  in  a  state  of  development  at  this  time,  but  currently  NCP 
Analysis  is  applied  to  sets  of  single-trial  time  series  in  about  25 
brief  analysis  intervals,  125  to  250  msec  wide,  for  up  to  64  BP 
ohanne I s , 

Several  prior  generations  of  NCP  Analysis  have  been  used  to  study  both 
simple  and  oomplex  tasks,  and  a  number  of  findings  have  resulted. 

These  results  suggest  that  both  strictly  local i zat ion i st  (‘single 
equivalent  dipole*)  and  equipotent ial ist  (‘hologram*)  views  of 
neurooogn I t i ve  prooesses  are  unrealistic.  Even  simple  tasks  are 
assooiated  with  rapidly  shifting,  complex  mosaios  of  regional  BP 
patterns,  and  a  more  appropriate  model  of  neurooogn i t i ve  processing 
might  be  a  distributed  computational  network  involving  continuous 
oommunioat ion  between  many  specialized  parallel  prooess ing  elements. 
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Our  research  goal  is  to  dovolop  oxpor imenta I  paradigms  and  analytic 
methods  to  maasura  and  modal  thasa  complex  processes. 

The  initial  thirty  months  of  this  projaot  ware  directed  toward 
advanoing  the  technology  of  recording  and  analysis  of  multi-channel 
data  obtained  during  perceptuomotor -cogn i t i ve  tasks.  The  following 
milestones  were  achievedi 

1)  Development  of  a  highly  controlled  6  seoond  bimodal  task  which 
manipulated  functional  relations  between  sensory,  association  and 
motor  cortical  areas  by  varying  the  expected  and  actual  sensory 
modality  of  numeric  stimuli  requiring  a  precise  isometric  finger 
response  and  resulting  in  feedbaok  about  response  accuracy  (Section 
11)1 

2)  Development  of  a  system  for  on-line  recording  of  up  to  64  BP 
channels  for  much  improved  spatial  sampling,  as  well  as  eye  movements 
(EOG),  muscle  potentials  (EMG),  heart  rate  and  respiration  signals 
(Seotion  IV, A); 

3)  Development  of  algorithms  and  software  to  perform  a  current  source 
density  (Laplacian)  transform.  This  converts  scalp  electrical 
potential  distributions  into  measures  of  current  flow  into  and  out  of 
the  brain  at  each  electrode  site.  The  effect  of  the  reference 
electrode  is  eliminated,  and  in  many  instances  localization  is 
improved  (Section  V.O)j 

4)  Expansion  of  the  AOIEEG  Analysis  system  to  a  64-channel  capacity. 
Automation  of  mu  1 1 i -stepped  analyses  of  500  megabyte  data  sets 
(Section  IV) { 

5)  Development  of  improved  eye-movement - f i I  ter i ng  algorithms  to 
remove  contamination  by  eye  movement  potentials  from  single-trial  data 
based  on  differences  in  spectral  signatures  between  eye  movements  and 
brain  potentials  (Section  V,E«1)«  Compilation  of  a  large  data  base  of 
artifacts  for  development  of  a  new  generation  of  automated  artifact 
rejection  algorithms  (Section  V,E,2)| 

6)  Development  of  improved  methods  of  averaged  evoked  time  series 
analysis,  inoludingi 

a)  A  simple  algorithm  for  improved  averaged 
event-related  potential  (ERP)  estimation  whioh  aohieves  a 
substantially  higher  s ignal - to-noise  ratio  on  fewer  trials  without  the 
imposition  of  a  priori  presumptions  about  the  shape  or  exact  time  of 
event-related  signal  components  (Seotion  V.B,1)( 

b)  Application  of  Wigner  Distributions  to  study 
split-seoond  changes  in  the  frequency  composition  of  evoked  time 
series.  This  is  the  first  application  of  this  method  to  ERPs,  and  it 
stay  prove  to  be  a  useful  researoh  tool  research,  complementing  the 
information  provided  by  traditional  time-domain  amplitude  analysis 
<8eot ion  V.B.2) | 
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7)  Development  of  improved  methods  for  single-trial  pattern 
recognition  analysis,  inoluding) 

a)  A  rapid  s i ng le-channe I  scanning  measure  to  select 
trials,  intervals  and  channels  with  task-related  signals  for  further 
analysis  (Section  V.A,1)| 

b)  A  statistical  ‘baseline*  estimation  procedure  which 
provides  a  nearly  ‘neutral'  comparison  data  set  against  which  to 
compare  event-related  patterns  (Section  V.A.2.b).  This  eliminates  the 
necessity  of  using  questionable  ‘resting  state*  or  prestimulus 
‘basal ines* ; 

8)  Trial  application  of  all  of  the  above  to  a  set  of  data  consisting 
of  about  SOO  trials  of  an  auditory  and  visual  perceptuomotor  task 
recorded  with  49  BP  channels  (Sections  III  k  V). 


2.  Prev i out  oj<per  imental  bindings  which  motivated  the 

lJtghniff.il  non  ts El r_imt.ru  1 1  developments  described  above 

NOTEi  It  must  be  understood  that  scalp-reoorded  potentials,  even 
unaveraged  time  series,  are  not  necessar i I y  oortical  in  origin.  Until 
the  issue  of  source  localization  is  better  understood,  conventional 
soalp  designations  for  electrode  positions,  which  refer  to  underlying 
oortioal  areas,  should  not  be  taken  to  imply  measurement  of  the 
activity  of  those  cortical  areas.  For  oonvenienoe,  we  use  the 
conventional  scalp  designations  subject  to  this  oaveat, 

a.  Complex  perceptuomotor  and  cognitive  activities  such 
as  reading  and  writing  produce  unique,  spatially  differentiated  scalp 
EEQ  spectral  patterns.  These  patterns  had  sufficient  specifioity  to 
identify  the  type  of  task  ( EEQ  Clin.  Neurophvs i o I .  47i693-703.  1979). 
These  results  agree  with  previous  reports  of  hemispheric 
lateralization  of  ‘spatial*  and  ‘linguistic'  processing. 

b.  When  tasks  are  controlled  for  stimulus,  response  and 
per f ormance- r e I ated  factors,  complex  cognitive  activities  such  as 
arithmetic,  letter  substitution  and  mental  rotation  have  identical, 
spatially  diffuse  EE6  spectral  scalp  distributions.  Compared  with 
staring  at  a  dot,  such  tasks  had  approx imate I y  10%  reductions  in  alpha 
and  beta  band  speotral  intensities  ( EEG  Clin.  Neuroohvsiol .  47! 
704-710,  1979)  So i ence  2031665-666,  1979).  This  reduction  may  be  an 
index  of  task  workload.  Since  no  patterns  of  hemispheric 

lateral i zat ion  were  found,  this  study  suggests  that  previous  and  most 
ourrent  reports  of  EEQ  hemispheric  lateralization  may  have  confounded 
eleotrioal  aotivity  related  to  limb  and  eye  movements  and  arousal  wi th 
those  of  mental  activity  per  se  (Science  207 i 1 003- 1 008 ,  1980). 

o.  Split-second  visuomotor  tasks,  controlled  so  that 
only  the  type  of  judgment  varied,  are  associated  with  complex,  rapidly 
shifting  patterns  of  inter -e I eot rode  correlation  of  single-trial  brain 
potential  timeseries.  Differences  between  spatial  and  numeric 
judgsMnts  were  evident  in  the  task-cued  prestimulus  interval.  Complex 
end  often  lateral ized  patterns  of  task-related  correlation  changed 
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with  split-seoond  rapidity  from  stimulus  onset  to  just  prior  to 
response,  et  which  time  there  was  no  difference  between  tasks  I  Sc i ence 
213t318*922,  1981),  This  suggests  that  once  tssk -spec i f i c  perceptual 
and  cognitive  processing  were  completed,  a  motor  program  common  to 

both  tasks  was  being  executed# 

» 

d#  Rapidly  shifting,  fooal  patterns  of  difference 
between  tasks  can  be  extracted  with  NCP  Analysis#  The  move  and  no -move 
variants  of  a  split-second  visuospatial  judgment  task,  which  differed 
in  expectation,  in  type  of  judgment,  and  in  response  (move  or 
no-move),  were  associated  with  distinctly  different  patterns  of 
single-trial  evoked  correlation  between  channels  (Science  220x97-99, 
1983|  Psvohophvs i o I oav .  1984,  In  press)#  These  results  concur  with 
neuropsyoho I og i ca I  models  of  these  tasks  derived  from  functional 
deficits  of  patients  with  localized  lesions#  They  suggest  that 
although  simple  peroeptuomotor  tasks  are  associated  with  a  complex, 
dynamic  mosaic  of  BP  patterns,  it  is  possible  to  isolate  foci  of 
maximal  differences  between  tasks#  It  is  clear  that  without  a 
split-second  temporal  resolution  it  is  not  possible  to  isolate  rapidly 
shifting  lateral izat ions  which  are  presumably  associated  with  stages 
of  peroeptual ,  oognitive,  and  efferent  processing# 

e#  The  focal  patterns  of  evoked  correlation  derived  by 
NCP  Analysis  significantly  distinguished  the  single-trial  data  of  7  of 
the  9  people  in  the  above  study#  This  suggests  that  consistent 
neurooogni t i ve  processes  were  measured  in  the  majority  of  right-handed 
male  research  participants  ( Psvohophvs i o I oov .  1984,  In  Press)# 


3#  Similarities  and  Differences  Between  Our  New  Experimental 
garafl j ami  Antlyita  snd  Convent  ional  £RjP  Techniques 

Our  experiments  and  analyses  are  grounded  on  the  body  of  information 
gained  from  ERP  research  and  have  the  same  underlying  goal:  to  resolve 
spatially  and  temporally  overlapping,  task-related  neural  processes# 
Our  approaoh  differs  in  several  ways  from  the  currently  popular  ERP 
paradigms  which:  a)  require  a  differential  judgment  between  task 
relevant  and  irrelevant  (usually  rare  and  frequent)  stimuli;  b)  then 
extract  a  few  features  from  averaged  ERPs  by  peak  picking  or  principal 
ooaiponents  analysis  (PCA);  and  c)  perform  hypothesis  testing  on  the 
f natures  with  ANOVA#  First,  because  neurocogn i t i ve  processes  are 
complex,  we  are  concerned  with  spat iotemporal  task-related  activity 
recorded  by  many  (currently  up  to  64)  electrodes  in  many  (currently 
about  25)  time  Intervals  spanning  a  six-second  period  extending  from 
before  a  oue,  through  stimulus  and  response,  to  presentation  of 
feedbaok#  By  oontrast,  most  ERP  experiments  are  concerned  with 
measurement  of  one  to  a  few  ’components'  associated  with  a  single 
stimulus  or  response  registration  in  a  few  channels  in  an  epoch  under 
one  second#  Additionally,  NCP  Analysis  quantifies  neurooogn i t i ve 
aotivity  in  terms  of  a  variety  of  parameters,  rather  than  just 
amplitude  and  latency  of  ERP  peaks#  This  increased  dimensionality  of 
parametr i zat ion  may  faoilitate  the  measurement  of  subtler  aspeots  of 
neurocogn i t ive  processes#  Seoond,  the  questionable  assumption  of  a 
multivariate  normal  distribution  of  BP  is  not  made  in  NCP  Analysis# 
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Third,  feature  extraction  and  hypothesis  testing  are  performed  as  a 
single  prooess  whioh  determines  the  differences  in  signal  properties 
between  the  conditions  of  an  experiment,  or  their  differences  with 
respect  to  a  'baseline*.  We  think  that  this  is  more  effective  than 
feature  extraction  techniques  such  as  PCA,  which  are  based  on  possibly 
irrelevant  criteria  such  as  statistical  independence  of  features 
regardless  of  discriminating  power.  Fourth,  in  NCP  Analysis 
task-related  patterns  of  consistency  are  extracted  from  sets  of 
single-trial  data.  Significant  results  may  be  obtained  as  long  as 
there  is  a  pattern  of  consistent  difference,  even  when  the  means  of 
the  two  data  sets  are  not  significantly  different. 

These  aspects  of  our  experimental  paradigm  and  NCP  Analysis  may  enable 
us  to  resolve  previously  unseen  neurocogn i t i ve  signals  from  the 
overriding  unrelated  background  activity  of  the  brain,  leading  to  a 
better  understanding  of  mass  neural  processes  of  human  goal -or  tented 
behaviors.  However,  this  approach  is  not  without  its  cost.  It 
currently  takes  2-3  technicians  to  run  an  experiment  with  64  channels, 
Manually  editing  this  data  for  artifacts  is  time  consuming  but 
necessary,  NCP  Analysis  requires  orders  of  magnitude  more  computing 
than  PCA  and  ANOVA,  Because  of  the  sensitivity  of  NCP  Analysis, 
highly  control  led  experimental  paradigms  are  required  to  assure  that 
the  results  are  truly  related  to  the  hypothesis  and  not  to  spurious 
factors,  (The  prooess  of  developing  one  such  task  is  described  in 
Seot ion  II,) 

Performing  these  experiments  and  analyses  requires  a  far  greater 
allocation  of  effort  and  resources  than  conventional  ERP  experiments, 
and  although  we  have  obtained  several  promising  results  with  previous 
generations  of  NCP  Analysis,  we  must  caution  that  'the  jury  is  still 
out*.  If  results  in  the  next  several  years  prove  that  this  new 
approach  is  really  worthwhile,  it  will  be  possible  to  optimize  and 
standardize  our  methods  and  analysis  for  application  by  other 
laboratories,  using  current  generation  h i gh-per f ormanoe 
supermi orocomputer  systems, 

4*  Evoked  Correlations  Between  Scalo  Electrodes 

For  the  past  few  years  we  have  concentrated  on  a  measure  of  waveshape 
similarity  (orosscor re  I  at i on )  between  time  series  from  pairs  of 
eleotrodes.  The  crosscorrelation  approach  is  based  on  the  hypothesis 
that  when  areas  of  the  brain  are  functionally  related  there  is  a 
consistent  pattern  of  waveshape  similarity  between  their 
maoropotent ial s.  There  are  a  number  of  considerations  in  interpreting 
correlation  patterns  of  scalp  recordings,  such  as  volume  conduction 
from  suboortioal  sources  and  driving  by  distant  generators.  Some  of 
the  asibiguities  may  be  mitigated  by  careful  experimental  design,  but 
the  neurophysiological  interpretation  of  correlation  patterns  is  an 
unsettled  issue.  We  are  engaged  in  collaborative  studies  using 
primate  models  and  using  MEG  recordings  to  address  this  issue.  The 
siethods  and  results  of  these  studies  will  be  described  in  future 
reports. 


9.  Current  Procedures  Used  in  NCP  Analysis 


November  13,  1384  <56> 


Ve  us*  the  generic  term  NCP  Analysis  to  refer  to  the  mul t i -stepped 

analysis  we  have  been  developing  to  extract  task-related 

spat iotemporal  patterns  from  the  unrelated  ‘noise*  of  the  brain. 

There  have  been  two  prior  generations  of  NCP  Analysis.  The  first 
measured  background  EE8  spectral  intensities  from  8  channels  while 
people  performed  complex  tasks,  suoh  as  arithmetic  problems,  lasting 
up  to  a  minute.  The  second  measured  orosscor relat ions  between  91 
pairwise  combinations  of  19  electrodes  recorded  during  performance  of 
simple  split-second  pereeptuomotor  tasks.  A  third  generation  is 
currently  nearing  completion  which  operates  on  up  to  64  channels 
reoorded  during  a  controlled  sequence  of  stimuli  in  which  a  person 
prepares  for  and  executes  peroeptual  judgment  and  motor  control  tasks 
and  reoeives  performance  feedback.  Beoause  of  the  large  size  of  these 
single-trial  data  sets  (up  to  100  megabytes  for  each  person),  several 
passes  through  the  data  are  required  to  oompiete  the  analysis.  The 
first  pass  selects  important  ohannels,  intervals  and  trials  to  reduce 
the  asiount  of  data  prior  to  measuring  interdependencies  between 
ohannels  (desoribed  in  Section  V.A  along  with  an  example  of  its 
application).  The  second  pass,  whioh  is  our  rent  I y  under  development, 
will  measure  interdependencies  and  tim.ng  relations  between  channels 
on  the  reduced  data  set. 

NCP  Analysis  essentially  oonsists  of  the  application  of  a 
layered-network,  nonlinear  mathematical  pattern  c lass i f i cat i on 
algorithm  (also  oal led  a  pattern  recognition  algorithm)  to  sets  of 
single-trial  brain  potential  features  in  order  to  extract  task-related 
signals  which  distinguish  two  sets  of  data.  The  sets  of  data  can  be 
two  different  experimental  tasks  (between- task  analysis),  or  they  can 
be  an  experimental  task  and  a  reference  ‘baseline*  data  set 
(within-task  analysis).  In  order  to  extract  the  most  spat iotemporal 
information,  the  pattern  recognition  algorithm  is  applied  many  times 
to  separate  groups  of  features  organized  aooording  to  their  temporal 
relation  to  a  stimulus  or  response  and  aocording  to  their  anatomical 
position  on  the  soalp.  Diagrams  are  then  drawn  in  a  head  outline 
format  showing  the  times  and  areas  with  task -related  information. 

In  the  seoond  generation  of  NCP  Analysis,  digital  lowpass  filtering 
was  used  for  between-ohannel  correlation  measures  due  to  the  low 
signal -to-noise  ratio  (SNR)  in  single  trials  (Bevins,  et  ai,  1981, 
19S3,  1984).  This  reduoed  the  susoept i b i I i ty  of  the  correlation 
measure  to  noise  but  smeared  temporal  relationships.  However,  the 
measurement  of  preoise  time  relationships  of  waveform  components 
between  ohannels  is  a  prime  objective  in  our  current  work.  Therefore, 
a  signal  enhancement  teohnique  was  required  to  improve  SNR  without  the 
time-smearing  effeot  of  conventional  filtering.  This  led  to  the 
development  of  a  first-pass  ‘soanning*  method  to  select  optimal 
subsets  of  s ignal -bear i ng  trials,  ohannels  and  intervals  to  improve 
the  SNR  for  oor relat ion -based  measures.  This  method  uses 
single-ohannel  filtered  amplitude  as  a  feature  for 

pat tern-reoogn I t ion.  Trials  without  consistent  task-related  signals 
are  rejected  if  the  pattern  recognition  algorithm  is  unable  to 
discriminate  them  from  sets  of  deta  known  to  have  a  low  event -related 
signal  (see  Section  V.A.1).  As  of  the  date  of  this  report,  the  second 
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pass  of  computing  orossoor rslat ions  botwoon  ohonnols  on  trials 
selected  in  tho  first  pass  has  not  yet  baan  avaluatad.  In  both 
passes*  saparata  analysas  ara  parformad  on  faaturas  computed  from 
129-290  msao  data  windows  iooatad  at  latanoias  datarminad  for  aaoh 
parson  from  tha  major  paaks  of  thair  avaragad  ERPs*  as  wall  as  from 
stimulus  and  rasponsa  initiation  times.  Howavar ,  tha  faaturas 
thamsalvas  ara  always  computad  on  single-trial  data* 

Thara  ara  17  major  stops  in  NCP  Analysis! 

1)  Racord  a  suffioiant  amount  of  data  using  up  to 
64  electrodes.  Computa  ourrant-souroa  dansitias  at  aaoh 

non-par iphara I  alaotroda  looation  from  tha  potantiai  distribution 
(this  mathod  is  currantiy  undar  avaluation). 

2)  Craata  a  singlai  ar t i faet-f raa  data  sat  which 
will  ba  comparad  with  a  rafaranoa  'basaiina*  data  sat.  Al tarnat i ve l y * 
oraata  two  data  sots  that  diffar  according  to  a  spaoifiad  or i tar  ion* 
such  as  tha  typa  of  expectation*  but  which  ara  otharwisa  balancad  for 
stimulus*  rasponsa  and  par formanca-ralatad  factors.  In  aithar  case* 
tha  two  data  sats  raprasant  conditions  which  ara  to  ba  oompared. 

3)  Computa  avaraga  st imulus-ragistarad  (and 
rasponsa-ragistarad  whan  appropriata)  tima  sarias  for  aaoh  person. 

4)  Usa  tha  latanoias  of  paaks  in  aaoh  parson's 
avaragad  tima  sarias  (ERP)  to  datarmina  tha  oantarpoints  of  analysis 
windows  for  singla-trial  pattarn  raoognition  analysis.  Adjust  tha 
window  siza  to  span  tha  paak  (usually  129  or  290  msao).  Sat  othar 
windows  at  stimulus  and  rasponsa  timas. 

9)  For  tha  first  soanning  pass*  low-pass  filtar 
tha  tima  sarias  at  7  Hz*  than  daoimata.  Usa  3-7  tima  sarias  valuas  as 
faaturas. 

6)  Partition  tha  data  of  tha  two  conditions  into 
thraa  training  and  (non-ovar lapping)  validation  sats. 

7)  For  aaoh  ohannel*  apply  a  nonlinear* 
layarad-natwor k  pattarn  raoognition  algorithm  to  ganarata  aquations 
whioh  raprasant  invariant  diffaranoas  in  tha  currant  at  tha  intarval 
ohosan.  Tha  ovaral I  parformanca  is  optimizad  to  maximize  tha  product 
of  tha  par formanoas. 

8)  Tast  tha  classification  aquations  with  tha 
validation  data  to  datarmina  whioh  alactrodas  diffar  significantly 
batwaan  oond i t i ons . 

9)  Rapaat  staps  7  and  8  for  aaoh  training  and 
validation  data  sat  and  computa  tha  maan  validation  sat  acouraoy. 

10)  8at  tha  (aval  of  significant  classification 
••II  abova  that  attainad  with  tha  data  randomly  assigned  to  two 
pseudo-oondi t ions. 
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11)  Draw  diagrams  showing  which  alactrodas  contain 
task  raiatad  information  in  aaoh  intarvali 

12)  For  aach  channel.  compute  avaraga  ERPS  and 
Vignar  Distributions  of  tha  avaraga  of  thosa  trials  with  task-ralatad 
signals  (i.e.  trials  corractly  classified  in  tha  validation  sat).  End 
of  pass  ona. 

13)  Baginning  of  pass  two.  Salact  intervals  and 
channels  for  measurement  of  spatial  i ntardapandanc i as  and  time  delays 
by  examination  of  tha  results  of  step  12. 

14)  Compute  multi  lag  cross  covariance  functions 
between  chosen  ohannals  and  many  other  channels  for  each  trial  in  each 
selected  analysis  window  on  tho«®  trials  with  task-related  signals. 

Use  a  measure  of  the  center  of  gt«.uity  of  the  cross  covariance 
function  as  a  feature  to  characterize  the  lag  time  between  channels. 

15)  Group  center  of  gravity  measures  into  anatomic 
constraint  groups  (also  called  “electrode  groups*).  These  represent 
the  time  lag  between  a  chosen  principal  eleotrodels)  and  a  number  of 
secondary  electrodes,  (e.g.  time  lags  between  the  left  occipital 
eleotrode  and  many  other  eleotrodes). 

16)  Repeat  steps  6-10  on  eaoh  electrode  group. 

17)  Tabulate  equations  of  significant  electrode 
groups.  Diagram  s i gn i f i oant I y  differing  principal  electrodes  and 
their  most  “prominent*  time  lags  with  other  electrodes  in  each 
analysis  interval.  (The  prominent  time  lags  of  a  significant 
eleotrode  group  are  determined  by  reoursion  over  steps  7-10). 


An  eighteenth  step  is  being  tested  wioh  computes  time-varying 
multivariate  models  of  the  spatio-temporal  neurocogn i t i ve  signals 
extracted  in  the  first  seventeen  steps. 

«•  Major  Obstacles  to  Progress 

•  •  fticah y-ijjbiJ  And  £nai aou-iifl  Qbius.l  u 

The  development  of  preoise  el ec t r ophys i o I og i ca I  methods  for  measuring 
brain  funotion  depends  on  technological  advances  in  several  areas. 

The  fundamental  problem  in  extracting  meaningful  information  from 
brain  electromagnet ic  signals  is  that  only  a  subset  of  the  brain's 
hundreds  of  simultaneously  active  major  systems  may  be  performing 
processing  related  to  a  particular  per oeptuomotor  or  cognitive 
funotion.  Further,  this  subset  seems  to  shift  rapidly  among  widely 
distributed  anatomical  areas*  and  the  diverse  geometric  orientations 
of  neural  generators  in  these  dispersed  systems  results  in  complex 
potential  and  magnetic  field  distributions  at  the  scalp.  It  is 
possible  to  determine  the  sources  of  these  distributions  only  in 
speoial  oases  whioh  meet  £  priori  assumptions,  such  as  an 
equi valent-single-dipole  generator  (Kavanaugh.  et  a  I . »  1978;  Sidman. 
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at  a I . ,  1978|  Dareey*  at  il . .  1980f  Wi I  I iamson  and  Kaufman*  1981a, b{ 
Kaufman  and  Williamson*  1982|  Romani  1982|  Okada*  1983j  Cohan  and 
Cuff  in*  1983|  Barth  1984),  Thasa  rastrietions  ara  unraalistio  in  tha 
oasa  of  higher  brain  functions*  which  involve  tha  integration  of 
processing  in  a  number  of  diserata  systems  (Luria  1970) •  in  such 
oases*  tha  ‘equivalent  dipole*  solution  will  not  ref  loot  tha  actual 
locations  of  several  s imu I tanaous I y-aot i va  sources. 

Furthermore*  tha  tissues  whieh  intervene  between  the  oortex  and  the 
soalp  greatly  distort  brain  potentials  (BPs)  by  acting  as  a  spatial 
low-pass  filter.  Low  temporal  frequenoy  BPs  are  spatially  smeared  and 
high  teaporal  frequenoy  BPs  are  attenuated  beoause  they  have  low 
spatial  ooherenoe  at  the  oortex  (Cooper*  Al  jJL,  *  1963),  Due  to 
variations  in  tissue  properties*  the  degree  of  distortion  varies  at 
different  points  on  the  soalp  and  is  different  for  different  people. 
This  distortion  oan  be  corrected  by  two-dimensional  spatial 
deoonvolut ion  (Nioholas  and  do  Loohe*  1 976 |  Doyle  and  Bevins*  In 
Prep,)*  but  measurements  of  the  thickness  and  oonductanoe  of  the 
intervening  tissues  at  eaoh  electrode  site  are  required  to  obtain 
aoourate  solutions  (Ary*  Al  Jj.,  *  1981),  Use  of  ultrasonic  devioes  to 
sieasure  skull  thiokness  is  problematic*  but  the  availablility  of 
nuo I ear  magnetic  resonanoe  (NMR)  scanners  should  improve  this 
si tuat ion. 

The  magnetic  fields  generated  by  the  brain  are  not  greatly  distorted 
by  intervening  tissues*  and  improvements  in  superconducting  quantum 
interference  device  (SQUID)  sensor  technology  are  eagerly  awaited. 
Substantial  progress  in  looating  single  equivalent  current  dipole 
souroes  has  already  been  made  with  the  oumbersome  single-channel 
8QUI08  currently  in  use.  The  work  of  Williamson*  Kaufman*  Romani  and 
Okada  at  N,  Y.  University  with  healthy  people*  and  that  of  Barth* 
Beatty*  8utherling  and  Engel  at  U«C«L,A»  with  epileptic  patients*  have 
set  the  standard  in  this  regard  (Williamson  and  Kaufman*  1981ab{ 
Kaufman  and  Williamson*  1982|  Okada*  1983|  Barth*  et  at*  1984 j  Richer* 
et  at*  1983|  Maolin*  et  al •  1983|  Romani*  et  al*  1982), 

Simultaneous  recording  of  many  electrical  and  magnetic  channels  is 
obviously  desirable  for  improving  souroe  localization, 

*>♦  Baa.i ff  SgjjtnfiJI  Obstacles 


There  is  a  more  fundamental  impediment  to  progress,  namely  that  the 
functional  signifioanoe  of  mass  brain  signals  is  not  known.  The 
opinion  of  most  neurophysiologists  is  that  maoropotent ial  patterns  are 
useful  epi phenomena  whioh  oan  supplement  the  information  gained  from 
reoording  the  all-or-nothing  aotion  potentials  of  individual  neurons, 
Aooording  to  this  view*  like  the  smoke  from  a  faotory,  maoropotent ial s 
stay  indioate  when  a  brain  region  is  processing,  but  cannot  provide 
store  specif io  information.  However*  this  popular  belief  is  challenged 
by  the  advanoed  studies  of  Freeman  and  his  assooiates  at  U«C« 

Berkeley,  These  studies  demonstrate  unequi vooal I y  that  stable 
siaoropotent ial  patterns  of  the  ol faotory  oortex  of  rabbits  embody 
eswrgent  properties  not  present  in  the  firing  of  single  neurons.  The 
spatial  asiplitude  patterns  of  these  neural  ‘searoh  i stages'  constitute 
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'templates'  of  specific  expected  odors  and  contain  specific 
information  about  the  odor  actually  perceived  (Freeman,  1975,  1979abc, 
1981,  1983),  Of  course,  it  is  a  large  leap  from  the  simpler 
paleocortex  to  the  more  complex  structure  of  the  neocortex  I  the  source 
of  most  sea  I p- recorded  activity  --  Petsche,  et  a  I . .  1984|  Speokmen,  e  t 
al  .  1984),  but  similar  general  principles  of  mass  neural  organization 
may  well  apply.  The  strict  ep i phenomena  I i st  view  is  being  challenged 
by  other  lines  of  advanoed  research.  It  has  been  suggested  that 
low- intensi ty ,  I ow- frequency  electromagnetic  fields  have  the 
capability  of  influencing  the  metabciic  activity  of  individual  neurons 
(review  in  Adey,  1981),  If  so,  such  fields  could  play  a  significant 
role  in  mass  neural  information  processing.  There  is  some  evidence 
for  the  existence  of  neural  processes  which  are  diffuse  and 
widespread,  rather  than  strictly  localized  as  in  traditional  models. 
These  processes  are  of  relatively  long  duration  compared  to 
single-neuron  firings,  and  their  larger  spatial  extent  enables  them  to 
exert  facilitory  and  inhibitory  influence  on  vast  numbers  of  neurons. 
These  processes  may  operate  by  large-scale  integration  of  classical 
synaptic  mechanisms,  or  may  involve  less  understood  modes  of 
commun  i  ca  t  i  on ,  such  as  the  serotomc  fibers  lining  the  walls  of  the 
cerebral  ventricles,  It  has  been  proposed  that  these  processes  are 
responsible  for  the  regulation  of  higher  level  behavioral  states  such 
as  arousal ,  reinforcement ,  and  memory  formation  (Dismukes,  1979), 
Macropotent ial s,  rather  than  single-neuron  firings,  must  be  studied  in 
order  to  determine  the  effects  of  these  systems, 

These  teohnical  obstacles  and  matters  of  basic  scienoe  are  currently 
being  attacked  by  a  number  of  interdiscipl inmry  teams  of  physical, 
medical,  biologioal  and  behavioral  scientists,  and  considerable 
progress  may  be  achieved  in  coming  years.  Advances  in  sensing, 
computing  and  digital  signal -processing  technologies,  combined  with 
carefully  controlled  experimental  designs,  are  propelling  this 
progress  at  an  ever  increasing  pace.  While  earlier  research  was  often 
character i zed  by  technically  naive  'tinkering',  or  blind  application 
of  new  analytic  technologies  without  adequate  attention  to 
neurophysiology  and  experimental  control,  this  current  work  reflects 
an  integrated,  interdisciplinary  approach, 


c.  Conceptual  Obitaclosi  Imp  I ici t  Brain  Mode  I s  ua  ££& 
iflti  EBP  Studies 

Current  paradigms  implicitly  assume  an  overly  simplistic  model  of 
brain  activity,  In  exploring  new  directions  for  research,  it  would 
be  useful  to  more  carefully  examine  the  assumptions  upon  which  these 
models  are  based,  In  EEG  studies,  it  is  assumed  that  the  brain 
essentially  consists  of  the  cerebral  neocortex  which  is  represented  by 
regions  cor  respond i ng  in  number  to  the  scalp  electrodes,  The  energy 
emitted  by  these  regions  in  several  frequency  bands  over  periods  from 
several  seconds  to  minutes  is  thought  to  be  related  to  the  number  of 
neurons  engaged  in  ’cooperative  processes'  (Elul,  1969,  1972ab>, 
Specific  higher  brain  functions  are  assumed  to  be  associated  with 
character i st i c  cooperative  processes,  which  may  be  conceptualized  as 
relatively  stable  macrostates  with  distinct  spectral  intensity 
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patterns)  pathological  conditions  manifest  as  alterations  of  these 
patterns  (Savins  and  Schaffer*  I960)* 

A  store  dynamic  model  underlies  ERP  studies.  In  clinical  paradigms 
whioh  test  the  integrity  of  sensory  pathways*  there  may  be  a  direct 
oonoeptuai  association  of  the  timing  of  successive  peaks  with 
asoending  anatomical  structures.  But  the  situation  is  more  complex  in 
studies  of  the  later  stages  of  information  processing.  In  these 
paradigsis*  many  of  whioh  require  performance  of  a  task*  it  is  assumed 
that  once  ‘programsied*  with  a  set  of  task  instructions*  the  brain 
operates  on  a  stimulus  in  more-or-less  sequential  information 
prooassing  stages  resulting  in  a  response  determini st ical I y  related  to 
the  stimulus.  The  successive  major  peak  complexes  and  low-frequency 
waves  of  the  averaged  time  series*  or  principal  components 
corresponding  to  them*  are  thought  to  correspond  to  these  stages. 
Task-related  changes  in  peak  voltage  or  component  amplitude  are 
considered  to  represent  changes  in  processing  load*  in  direct  analogy 
to  reductions  in  background  EE6  alpha  band  spectral  intensity  during 
task  performance.  Differences  in  the  scalp  voltage  distribution  of  an 
ERP  peak  or  component  are  interpreted  as  reflecting  the  presence  of 
distinot  neural  generators.  Shifts  in  peak  or  component  timing* 
particularly  of  the  various  N2  and  P3  components*  appear  to  correspond 
with  ohangas  in  the  timing  of  certain  ‘endogenous*  mental  processes* 
but  the  contribution  of  tr ial -to-tr ial  variability  to  these  latency 
shifts  is  most  often  not  explicitly  measured*  nor  is  the  deconvolution 
of  spatially  overlapping  processes  explicitly  attempted.  While  ERP 
experiments  continue  to  generate  interesting  results*  the  fundamental 
problem  is  that  the  relations  between  ERP  peaks  and  brain  functions 
are  not  aotually  as  direot  or  as  simple  as  they  are  assumed  to  be 
(Bavins*  1983a*  1984al .  Sinoa  the  same  ERP  effects  can  be  produced 
under  a  variety  of  oircumstances*  and  since  minor  variations  in 
experimental  manipulations  oan  result  in  different  ERP  effects*  one 
must  oonolude  that  some  of  the  effects  have  no  intrinsic  existence 
outside  the  context  of  a  particular  experimental  paradigm. 

Further  improvements  in  event  -  re  I ated  signal  estimation  can  be 
expaoted  when  analytio  models  are  used  with  greater  neurophysiological 
fidelity.  Classical  models  assume  that  physically  distinct  brain 
systems  produoe  event  -  re  I ated  signals  and  unrelated  background  EEG 
aotivity.  In  actuality*  the  activity  pattern  of  a  neural  system  seems 
to  undergo  a  reorgan i zat i on  as  some  subset  of  the  system's  neurons 
becomes  involved  in  event-related  processing.  The  summated 
extracellular  brain  potentials  may  manifest  this  reorgani zat ion  by 
changes  of  amplitude*  frequency*  morphology  and  timing  characteristics 
of  low  frequency  waveforms.  A  more  realistic  analytic  model  may  thus 
be  a  hybrid  of  the  ‘post -st imu I  us  additive  noise*  and  *pre-  to 
post -st imu I  us  transformat  ion'  approaches.  In  this  case*  the 
multichannel  event -related  signal  would  be  modeled  by  a  nonlinear* 
time-varying  transformation  of  the  pre-stimulus  brain  state  plus  an 
event-related  prooess*  plus  random  noise.  The  pre-stimulus  brain 
state*  a  funotion  of  a  person's  expectations*  would  itself  be  modeled 
by  a  time-varying  signal  plus  an  additive  noise  process.  A  further 
eomplioation  is  that*  at  least  in  the  instance  of  higher  cognitive 
funotions*  the  brain  does  not  necessarily  behave  deterministically. 
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Beoeuee  of  the  h i eraroh i oa I ,  parallel  organization  of  the  brain,  and 
because  of  its  adaptive  learning  capabilities,  the  same  behavior  can 
result  from  a  variety  of  different  processes  involving  different 
neural  systems.  In  attempting  to  extract  f unc t ion- re  I ated  signals,  it 
would  seem  imperative  to  allow  for  this  multiplicity  of  possible 
siechan  i  sms . 

While  they  oontinue  to  be  fruitful,  both  currently  popular  approaches, 
EE8  and  ERP,  are  based  on  highly  oversimplified  models  of  the  brain 
and  information  processing.  The  former  has  insufficient  temporal 
resolution  to  resolve  f ract ion-of -a-second  processes,  while  the  latter 
performs  an  excessive  data  reduction  resulting  in  too  few  parameters 
to  oharacterize  complex  processes.  Although  still  exceedingly  crude, 
a  somewhat  more  realistic  research  model  for  the  next  generation  of 
analytic  studies  may  be  to  consider  the  brain  as  a  local  distributed 
computat i onal  network  (Gevins,  1981,  1983b,  1984).  A  first  step  in 
esipirioally  developing  such  a  model  will  be  to  attempt  to  measure  the 
input  and  output  of  each  of  a  number  of  simultaneously  active  major 
processing  nodes  and  the  rapidly  shifting  temporal  patterning  of 
communication  between  them  during  simple  tasks. 


II.  gJLQI.LMfl  OF  AN  AUP I  TORY-V  I  SUAL  PERCEPTUOMOTOR  TASK 
A.  Design  Considerations 

A  number  of  issues  were  considered  in  designing  a  bimodal  paradigm 
sufficiently  controlled  to  reveal  NCPs  which  distinguish  auditory  and 
visual  peroeptuomotor  tasks  during  the  modality-cued  prestimulus  and 
post  stimulus  intervals. 

1 .  Cgfln i n vt  Cpnudiret ioni 

Two  hypotheses  are  being  tested.  First,  NCPs  should  show 
neuroanatomi ca I  I y  i nterpretab I e  differences  between  the  processing  of 
auditory  and  visual  numeric  stimuli  in  post  stimulus  intervals  when 
fsature  extraction  is  thought  to  occur  in  sensory  and  related  cortical 
areas.  There  should  be  minimal  differences  after  sense-specific 
processing  is  completed.  Second,  NCPs  should  differ  in  the 
modality-cued  pre stimulus  interval  as  a  function  of  the  preparation  to 
reoeive  either  visual  or  auditory  stimuli,  The  second  hypothesis 
requires  complete  equivalence  of  oue  properties  and 

per formance- re  I ated  faotors  between  auditory  and  visual  conditions, 
and  also  a  strong  inference  that  a  moda I i t y -spec i f i c  expectancy  set 
exists  in  the  cued  prestimulus  interval. 

2.  Exoer i mental  Controls 

The  first  hypothesis  lie.  post  stimulus  processing)  requires  control 
of  stimulus,  response  and  per formanoe- re  I ated  factors  across 
oonditions  so  that  the  major  difference  between  conditions  is  stimulus 
stodality.  Visual  and  auditory  modalities  have  fundamental 
differences.  Input  for  the  former  is  parallel  (for  brief  foveally 
presented  stimuli),  while  for  the  latter  it  is  serial.  Inherent 
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differences  in  auditory  and  visual  processing  I ateno i es  can  be 
ooaq>ensated  for  by  centering  several  analysis  windows  on  the  average 
ERF  peak  latencies  in  each  person  for  auditory  and  visual  conditions 
separately.  Differences  in  ERP  amplitudes  can  be  explicitly  measured . 
With  regard  to  the  seoond  hypothesis  (ie.  prestimulus  attentionai 
set)*  the  moda M ty-oued  paradigm  elioits  a  contingent  negative 
variation  (CNV)  between  cue  and  stimulus*  with  consequent  resolution 
after  stimulus  presentation*  NCP  Analysis  will  be  applied  to  measure 
pre-  and  post -st imu I  us  modality  differences*  The  results  may  shed 
some  light  on  the  interaction  between  preparatory  activity  and 
post-st imu I  us  processes* 

a*  Manma 

In  the  pilot  recordings*  a  montage  of  21  scalp  electrodes  positioned 
over  auditory*  visual*  motor*  parietal  and  dorsolateral  prefrontal 
oortices  was  used*  Four  additional  channels  recorded  vertical  and 
horizontal  EOS*  EM6  and  the  response*  Formal  recordings  employed  a  49 
ohannel  montage  (Figure  3b)  in  order  to  achieve  a  resolution  of 
approx imatel y  10  square  centimeters*  A  recent  study  of  trimodal 
attentionai  set  using  the  regional  cerebral  blood  flow  technique 
(Roland*  1982)  revealed  that  the  spatial  patterns  of  focal  neural 
aotivation  related  to  attention  and  modality  processing  are  complex 
and  finely  detailed*  even  with  a  30  sec  temporal  resolution*  For 
adequate  sampling  of  brain  potentials  cor  respond i ng  to  these  regions 
of  fooal  aotivation*  a  dense  ooverage  of  lateral  prefrontal*  superior 
and  inferior  posterior  parietal*  and  superior  and  posterior  temporal 
areas  is  required* 


8*  Task  PfVflqpmint  snd  Pi  lot  £fi£  Recordings 

i*  Phifi  flaa  l£_».  ii-ftj  t  (Btiftami  is.  mltauaii 

a*  Rat i onale.  The  existence  of  a  modality-specific 
prestimulus  attentionai  set  was  investigated  with  the  'miscueing* 
technique  (Posner*  1978)*  In  this  method  a  randomly  ordered  20%  of 
the  modality  cues  (a  visually  presented  letter  in  both  conditions)  are 
ineorreot*  The  lengthening  of  mean  response  time  in  these  miscued 
trials  is  considered  the  ‘cost*  incurred  by  the  expectation  of  a 
stimulus  in  a  speoifio  modality*  and  is  used  to  infer  the  existence  of 
a  modal i ty-speoi f io  preparatory  set* 

b»  Task .  Stimulus  presentation  and  response 
measurement  were  performed  by  the  real  time  subsystem  of  the  ADIEEG 
system*  whioh  also  digitized  the  25  ohannels  of  physiological  signals 
(21  soalp)  at  256  samples/sec*  The  participant  (P)  was  instructed  to 
fixate  a  point  at  the  center  of  the  CRT  screen  of  an  AED  II  graphics 
terminal  and  await  a  visually  presented  modality  cue  (V  for  visual;  A 
for  auditory;  duration  375  msec).  1.5  sec  later  the  stimulus  was 
presented*  Auditory  stimuli  consisted  of  the  numbers  1  to  9  generated 
by  a  Votrax  speech  synthesizer  and  presented  through  two  speakers 
about  70  cm  above  the  participant's  head*  Duration  varied  from  245  to 
430  msec  (the  number  7  was  generated  as  *SEVN‘).  Visual  stimuli  were 
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single  digit  numbers  presented  on  the  CRT,  subtending  a  visual  angle 
of  under  one  degree.  Their  duration  was  equated  to  that  of  their 
corresponding  auditory  numeric  stimuli. 

The  participant  was  instructed  to  attend  the  modality  cue  and  'focus 
his  attention*  on  the  speakers  or  screen  as  indicated  by  the  cue, 
while  maintaining  his  gaze  on  the  screen.  He  was  to  respond  to  the 
stimulus  without  hesitation  with  a  ballistic  contraction  of  his 
right-hand  index  finger  on  a  modified  Grass  force  transducer,  exerting 
a  pressure  corresponding  to  the  stimulus  number  on  a  linear  scale  of  1 
to  9,  Feedback  indicating  the  exact  pressure  applied  was  presented  as 
a  2  digit  number  on  the  CRT  screen  (duration  375  msec)  1  second  after 
completion  of  response  as  determined  by  the  program. 

If  the  response  was  sufficiently  accurate,  the  feedback  number  was 
underlined,  signifying  a  'win*.  The  error  tolerance  for  a  ‘win*  was 
continually  adjusted  throughout  the  session  as  a  moving  average  of  the 
aocuracy  of  the  preceeding  five  correctly  cued  trials  in  visual  and 
auditory  modalities  separately, 

A  random  18X  of  the  trials  were  miscued  ( ie,  the  stimulus  arrived  in 
the  wrong  modality).  The  participant  was  to  respond  to  these  just 
like  the  oorreotly  cued  trials.  Correctly  cued  and  miscued  auditory 
and  visual  trials  were  presented  in  randomly  ordered  blocks  of  17 
trials,  sal f - i n i t iated  by  the  participant, 

o.  Record i nos ,  Eight  normal,  right-handed  adult  male 
participants  were  recorded  in  pilot  sessions  consisting  of  about  390 
to  700  trials  each.  The  electrode  montage  was  Fz,  F7 ,  FB,  aFI,  aF2, 
aCz,  Cz,  C3 ,  C4 ,  C9 ,  C6,  Pz,  P3,  P4,  T3,  T4,  T5,  T6,  Oz,  aOl  and  a02, 
referenced  to  linked  mastoids  (modified  expanded  10-20  system 
nomenclature!  see  Fig,  3a),  Vertical  and  horizontal  eye  movements, 
response  muscle  activity,  and  force  transducer  output  were  also 
recorded.  All  signals  were  amplified  by  a  64-channel  Bioelectric 
Systems  Model  AS-64P  amplifier  with  ,10  to  100  Hz  passband, 
continuously  digitized  to  11  bits  at  256  samples/sec  and  stored  on 
digital  tape.  Signals  were  also  recorded  on  three  8-channel 
polygraphs  to  monitor  the  session  and  for  off-line  artifact  editing. 
Average  response  times  (RT)  and  error  rate  (proportion  of  ‘lose* 
trials)  were  oomputed  for  behavioral  evidence  of  a  'cost*  due  to  a 
prestimulus  attentional  set  in  a  total  data  set  of  3735  trials, 

d.  Results  and  Discussion,  For  correctly  cued  trials, 
the  mean  response  times  were  quicker  for  visual  stimuli  for  all  but 
two  people.  Average  RT  across  P's  was  675  msec  for  visual  stimuli  and 
711  msec  for  auditory.  The  longer  RT  for  auditory  stimuli,  which  is 
opposite  to  the  findings  in  simpler  bimodal  paradigms,  may  be  due  to 
the  nature  of  the  verbally  presented  number  stimuli.  All  information 
needed  for  visual  stimulus  deooding  appeared  on  the  screen  within  33 
msao,  while  auditory  information  was  not  completed  for  up  to  several 
hundred  msec.  The  longer  RT  for  the  auditory  stimuli  may  also  be  due 
to  the  use  of  synthesized  speeoh  stimuli. 
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Table  1  -  Average  response  time  (in  msec)  and  standard  deviation  for 
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Tife.Lt  2  - 

Error  rate  (%  of  ‘lose’  trials)  in  correctly  cued  and 
miscued  auditory  and  visual  conditions  in  phase  one  of  task 
developments  (Miscued  trials  -  18%). 
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final  atacaad  condition  with  napes**  (p#7,  31  trial*) 
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In  the  miscued  trial*  a  lengthening  of  RT  and  increase  in  arror  rata  was 
obsarvad  in  almost  avary  oasa  (Tablas  1  and  2).  For  miscuad  auditory 
stimuli  tha  avaraga  inoraasa  in  RT  was  47  msac  and  tha  average  increase  in 
arror  rata  (proportion  of  'lose'  trials)  was  9 X,  For  miscuad  visual 
stimuli  tha  ’costs*  of  misouaing  ware  slightly  greater  (increase  in  RT  =  65 
msao|  inoraasa  in  arror  rata  ■  10%) .  Thera  was  a  small  (18  msac)  asymmetry 
in  RT  affa ct.  That  is*  misouaing  a  visual  stimulus  caused  a  greater 
increase  in  RT  than  miscuaing  an  auditory  stimulus.  This  asymmetry  and  the 
magnitude  of  tb#  RT  langthaning  in  miscuad  trials  is  in  agreement  with 
previous  studies  using  simpler  tasks  (Posner*  1978).  Further*  the  standard 
deviations  for  all  ouad  and  misouad  conditions  ware  similar  within  parsons* 
indicating  that  tha  RT  ’costs*  due  to  miscuaing  ware  based  on  a  consistent 
affect  rather  than  greater  variability  in  a  smaller  sample  (about  4  to  1 
ratio  of  correctly  vs*  incorrectly  cued  trials*)  Thus  it  was  verified  that 
there  was  a  ’cost*  in  tha  miscuad  trials  indicative  of  an  attantional 
commitment  to  a  particular  modality  in  tha  prastimulus  interval. 

Tha  ERPs  for  one  parson  (PI7)  are  shown  in  Figures  la  and  1b.  Tha  major 
difference  between  tha  correctly  cued  and  miscuad  waveforms  was  in  tha  P3 
peak.  In  tha  auditory  condition  (Figure  la)*  tha  miscuad  auditory  stimuli 
produoad  an  augmantat ion  of  tha  P 3  peak  at  314  msac.  This  P 314  was  maximal 
at  tha  midlina  parietal  electrode*  and  extended  to  midlina  f ronto-cant ra I » 
lateral  central  and  lateral  parietal  electrodes.  In  tha  visual  condition 
(Figure  1b)*  tha  miseuuad  stimuli  elicited  an  augmentation  of  the  P3  peak 
at  300  msec.  The  visual  P300  was  broader*  and  more  anteriorly  distributed 
than  the  auditory  P314.  A  pre-stimulus  CNV  is  evident  as  a  f ronto-cant ra I 
negativity.  Its  resolution  could  occur  at  different  latencies  after  visual 
and  verbal  stimuli.  Also*  the  information  content  of  the  verbal  stimuli 
occurred  at  various  latenoies  (as  in  ‘six*  and  ’seven**  or  'four*  and 
‘five**  as  oompared  to  the  other  numbers).  What  effect  this  may  have  upon 
the  latenoies  of  endogenous  ERP  components  is  not  known.  These  issues  will 
be  addressed  in  the  formal  study. 


2.  Phase  1 


19-  12)  :  IfcLfi  r.tuppntt  JLfi 


•  *  BaiianuL* 


The  behavioral  results  of  phase  one  confirmed  the  existence  of  a 
prestimulus  attentional  set.  In  phase  two  a  move/no-move  design  was 
employed!  the  ‘response*  to  a  miscuad  stimulus  was  to  make  no  movement. 
This  was  done  so  that  there  would  be  confirmation  of  attention  to  the  cue 
i n  each  trial. 

b.  Task .  The  task  was 
participant  was  instructed  to  make  no 
20%)*  A  monetary  incentive  was  added 
to  the  aoouraoy  of  response  (about  5  cents  for 
monetary  reward  was  displayed  at  the  end  of  each 
with  the  average  error  tolerances  (performance 
visual  trials.  The  oue- to-s t imu I  us  interval  was 
1  seo  for  PI12.  For  P*12  a  separate  run  was 
prestimulus  interval  for  the  visual  modality  onlyi 

22 


the  same  as  before*  except  that  the 
response  on  miscued  trials  (random 
by  rewarding  ‘win*  trials  according 
each  win).  The  accrued 
block  of  17  trials*  along 
index)  for  auditory  and 
2.5  sec  for  P's  19-11*  and 
recorded  with  a  2.5  sec 
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c *  Reoor d i noi .  Four  normal ,  right-handed  adult  males 
participated  in  sessions  consisting  of  about  100  practice  and  400  test 
trials.  The  21  channel  montage  consisted  of  Fz,  F 7 ,  F8,  aFI,  aF2«  aCz,  Cz, 
C3,  C4»  C5i  C6,  Pz,  P3,  P4»  aP5 »  aP6 »  T5,  T6»  Oz  ,  aOI  and  a02i  referenced 
to  linked  mastoids.  Tin  scalp  electrodes  were  attached  using  a  stretchable 
nylon  cap  (Electrocap  International).  Vertical  and  horizontal  eye 
mo v erne n t s  and  response  muscle  potentials  were  also  recorded.  Other  aspects 
of  the  recording  procedure  and  ERP  computation  were  the  same  as  phase  one. 
Cue-registered  averages  were  also  computed  for  PI12  for  the  cue- to-st imu I  us 
interval . 


d.  Results  and  Discussion.  At  the  2.5  sec  cue- to- s t imu I  us 
interval  (P's  §9-11),  there  was  a  high  rate  of  data  attrition  (about  25X) 
in  the  no-move  (miscued)  trials  due  to  mistaken  overt  responses.  At  the  1 
sec  interval  (P§12)  there  were  no  mistaken  responses  to  no-move  (miscued) 
stimuli.  The  high  rate  of  incorrect  responeses  to  no-move  stimuli  at  the 
2.5  sec  interval  may  have  been  due  to  decay  of  the  attentionai  set. 
Therefore,  the  1  second  interval  seems  more  desirable  for  the  study  of 
prestimulus  expectancy. 

The  st imu I  us- reg i stered  ERPs  for  the  move  trials  (Fig.  2a  and  2b)  were 
similar  to  the  correctly  cued  trials  of  Phase  one.  In  P's  §9,  10  and  12 
the  no-move  (misoued)  trials  elicited  an  augmentation  of  the  P3  peak.  The 
P3  peak  amplitude  in  both  move  and  no-move  averages  was  maximal  at  midline 
f ronto-oentral  sites.  A  similar  anterior  distribution  of  P300  peak 
amplitude  to  no-go  stimuli  was  reported  in  a  go/no-go  paradigm  with 
equiprobable  stimuli  (Simpson,  at  al,  1977). 

A  pre-stimulus  CNV  was  seen  in  all  participants.  In  a  cued  paradigm  such 
as  this,  a  CNV  may  well  be  ooncomitant  with  the  expectancy  set  we  wish  to 
study  (reviewed  by  Teooe,  1972).  CNV  negativity  was  largest  at  midline 
f ronto-cent ra I  eleotrodes.  Although  its  topography  did  not  seem  to  differ 
with  expected  stimulus  modality,  NCP  Analysis  of  the  cue- to-st imu I  us 
interval  in  the  formal  study  may  shed  more  light  on  the  important  issue  of 
the  existence  of  a  modality-specific  effect  on  the  anatomical  distribution 
of  CNV-related  aotivity,  an  issue  which  has  not  been  thoroughly  examined 
(Ri tter ,  et  al ,  1980) . 

3.  pint l  Paradigm 

A  screening  program  was  used  to  seleot  and  train  candidate  participants. 
During  screening  sessions  EEGs  were  recorded  from  Fz,  Cz,  Pz ,  aOI  and  a02, 
and  from  T1  and  T2  to  assess  EMG  activity  potentials  from  temporalis 
musoles.  Vertical  and  horizontal  eye  movements  were  also  recorded. 
Behavioral  records  were  examined  to  assess  the  quality  of  task  performance, 
polygraphs  were  inspected  to  determine  the  amount  of  data  attrition  due  to 
ar tifaot,  and  average  ERPs  were  oomputed  to  verify  the  presence  of  expected 
ERP  peaks.  About  one-third  of  the  candidates  screened  were  selected  for 
the  formal  study. 

As  of  1  JUN  84,  four  full  recordings  have  been  made  with  the  49  electrode 
montage  (Fig,  3b).  (The  64  channel  recording  cap  shown  in  Fig.  3c  is  now 
operat ional , )  Approximately  800  trials  were  recorded  from  each 
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Expanded  10-20  system  as  detailed  by  Plcton  et  al.  (1978). 
The  nomenclature  for  coronal  plane  sites  has  been  modified 
for  consistency.  Asterisks  indicate  special  placements. 
Circled  numbers  are  approximate  location  of  Brodmann  areas. 


Figure  Set 


Approximate  oortioei  areas  underlying  soelp  electrodes* 


Area  Bro 

striate  oortex  (80 

peristriate  oortex  (PSO 

peristriate  oortex  (PBO 

post*  inferotemporal  oortox  (ITC) 

prefrontal  oortex  (RFC) 

prefrontal  oortox  (RFC) 

sup*  temporal  oortex  (STC) 

inferopar ietal  oortex  (IRC)  (oaudal) 

inferopar letal  oortex  (IRC)  (rostral) 

sup*  parietal  oortex  (SRC)  (oaudal) 

sup*  parietal  oortex  (8PC)  (rostral) 

presMtor  oortex 

pre-oantral  gyrus 

post-oentral  gyrus 

primary  somatosensory 

frontal  eye  field 


idman  1 

17 

OZ 

18 

01  *  eoa 

18 

aOz*  e01»  a08*  IS*  R9 

37 

T5 

8 

Fi ,  FI*  F8»  aFI »  aFx 

10 

Fpz,  Fp 1  *  aFx*  aFI*  aF3 

22 

T3»  ePS  (oaudal  end) 

38 

aPS*  PS.  P3 

40 

aRS*  aP3,  CS 

7 

Rz.  PI,  P3.  aFI .  aP3 

S 

aRz*  aPI* 

6 

aCz*  aCI*  aC3*  aCS*  Cl 

4 

Cz*  Cl,  C3 

*2 

C3.  aPz 

8 

same  as  for  1  and  2 

• 

Fz,  FI.  aC3 

•  Using  the  data  of  Tailairaoh  and  Sxilila  1867. 

*  Left  side  listed  for  oonvenienoei  add  1  to  odd  numbered  eleotrodes  for 
homologous  right-sided  placements.  Eleotrode  nomenclature  smdified  from 
Rioton  et  al.  (1876)  for  oonsistenoy.  Starting  from  the  frontal  pole*  the 
ooronal  plane  arreys  are  designated  Fp,  aF  (anterior  frontal)*  F,  aC*  C, 
eR»  P*  aO*  0*  See  Fig*  8ai  Expended  18-20  System. 
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we II -prect i ced  perticipant  using  the  paradigm  and  recording  procedure 
described  above.  The  cue* to-st imu i us  interval  was  1  sec,  the  visual 
stimuli  were  thickened  and  increased  in  size  to  just  under  a  2  degree  • 

visual  angle,  and  the  proportion  of  miscued  (no-go)  trials  was  increased  to 
22%,  Current  Source  Oensity  (CSD)  derivations  were  computed  off-line  (see 
Sect  ion  V.D, 1 ) , 


III,  Pilot  Anal  vs i s  of  Data  from  Final  Par  ad i om  • 

A.  Format  ion  of  Data  Sets 

The  data  of  one  person  (AV07)  was  used  to  refine  the  signal  processing  and 
analytic  methods  to  be  used  in  the  formal  analysis  (see  Section  V),  After 
inspection  of  polygraphs  and  behavioral  records  to  eliminate  artifacted  and  • 
incorrectly  performed  trials,  an  edited  data  set  of  S10  trials  remained. 

These  trials  were  submitted  to  the  ADISORT  program  (see  Section  IV. 6)  to 
form  6  pairs  of  data  sets:  s t imu I  us- r eg i s tered  auditory  vs.  visual  move, 
auditory  move  vs,  no-move,  visual  move  vs,  no-move,  and  auditory  vs.  visual 
no-move|  response- reg i stored  auditory  vs,  visual  mov«|  and  cue-registered 
auditory  vs.  visual  trials,  These  6  pairs  of  data  sets  were  each  examined  • 
for  the  following  differences  in  stimulus,  response  and  per f ormance- re  I ated 
variables:  magnitude  of  stimulus  number,  response  onset  time,  response 

movement  force,  velocity,  acceleration  and  accuracy,  adaptive  error 
tolerance  measure,  'arousal*  indices,  and  vertical  and  horizontal  EOG 
indices.  There  was  no  significant  difference  within  any  of  the  pairs  <-*f 

data  sets,  except  for  the  prestimulus  arousal  index  (p(,05)  in  the  audi  ory  • 
vs,  visual  move  pair,  which  may  reflect  a  differential  modal i ty-speci , ic 
expectancy,  and  the  response  time,  which  was  73  msec  shorter  for  visual 
than  for  auditory  move  trials,  Only  a  slight  amount  of  pruning  was 
required  to  balance  the  EOG  indices  so  that  the  pairs  of  data  sets  did  not 
differ  at  a  conservative  alpha  of  ,2,  Response  times  were  pruned  at  +2.5 
standard  deviations,  resulting  in  response  times  of  892+-  127  msec  for  "# 

auditory  and  817+-145  msec  for  visual  move  trials.  The  final  sorted  data 
sets  were  balanced  for  all  variables  (except  RT I  at  an  alpha  of  .2,  and 
consisted  of  149  auditory  and  215  visual  move  trials,  and  65  auditory  and 
74  visual  no-move  trials  for  the  s t imu I  us - r eg i s tered  sets.  The 
cue-registered  sets  consisted  of  119  aud i tory-cued  and  180  visual-cued 
trials, 

B.  Examination  q±  Averaged  Waveforms 

Averaged  ERP  and  Current  Source  Density  (CSD|  see  Section  V.0.1)  waveforms 
were  computed  from  the  balanced  data  sets  for  al I  channels  and  tasks 
(Figures  4  and  5,  and  20a,  21a,  22a  and  23a).  The  major  event  -  re  I ated 
peaks  were  first  determined  from  the  ERP  waveforms,  and  then  compared  with 
the  CSD  waveforms.  The  CSD  waveforms  exhibited  peaks  at  the  same  latencies 
as  the  ERPs,  but  their  amplitude  distributions  differed  greatly,  (CSD 
distributions  in  ‘head  format*  are  shown  in  Figures  14  -  16.)  CSD  peaks 
were  typically  more  spatially  localized  than  cor  respond i ng  ERP  peaks. 

Since  a  computed  common  average  reference  was  used  in  computing  the  ERPs  • 
for  this  participant  (AV07),  the  topographies  of  ERP  peaks  differed  from 
those  usually  found  in  *of f - the-head "  referential  recordings,  and  thus  will 
not  be  discussed  here,  CSD  peaks  are  labeled  according  to  the  latency  and 
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polarity  at  tha  channel  of  maximum  amplitude  (eg.  'Pi  17').  Positive 
amplitude  in  the  CSO  derivation  indioates  exiting  current  ('source')  and 
negative  values  entering  current  ('sink').  Peripheral  electrodes  will  not 
be  considered  (see  Section  V.O).  In  considering  the  topographies  of  CSO 
peaks,  it  should  be  remembered  that  since  the  CSO  is  a  measure  of  the 
ourvature  (second  derivative)  of  electrical  potential  at  the  scalp,  the 
maximum  amplitudes  of  CSO  peaks  occur  at  regions  where  the  curvature  of 
potential  is  greatest.  Por  example,  maximum  CSO  amplitude  for  the  CNV 
occurs  at  anterior  parietal  sites,  where  the  change  in  the  gradient  of 
potential  is  greatest. 

1 .  Exogenous  Peaks 

NIOOi  Visual  stimuli  elicited  a  sharp  peak  (P117;  Figure  4),  positive 
(’current  out’)  at  posterior  parietal  and  occipital  sites  (maximum  at  aOz ) , 
and  negative  (’current  in')  at  anterior  locations  (maximum  at  Cz  and  aPI). 

A  slight  lateral izat ion  was  present,  the  positive  peak  being  larger  at 
right  posterior  sites  and  the  negative  peak  at  the  left  anterior  parietal 
(aPI)  electrode.  The  negative  peak  was  smaller  and  less  localized  at  the 
soalp.  Auditory  stimuli  elicited  an  N109  peak  with  negative  amplitudes  at 
most  frontal,  central  and  anterior  parietal  sites  (Figure  5).  Maximum 
negativity  was  at  C2  and  aP4,  resulting  in  a  right -sided  lateralization.  A 
corresponding  positive  peak  was  evident  only  at  F5  and  auz » 

P200 i  A  second  peak  occurred  at  172  msec  for  both  visual  and  auditory 
stimuli.  The  enter i or -poster i or  polarity  was  approx imate I y  opposite  to  the 
preceding  peaks  in  each  modality.  The  visual  P172  was  positive  at  most 
frontal  and  anterior  central  sites,  with  maximal  amplitudes  at  aCz  and  aC3 
(Figure  4),  A  distinct  negative  peek  was  seen  only  at  the  anterior 
occipital  (aOz)  and  right  parietal  (P4  and  P6)  electrodes.  The 
distribution  of  the  auditory  P172  was  similar  to  the  visual  peak,  but  its 
amplitudes  were  slightly  larger  (Figure  5).  The  resolution  of  the 
prestimulus  CNV  also  occurs  within  this  latency  range,  and  probably  affects 
the  amplitudes  and  topography  of  this  peak. 

2.  Endogenous  Pfjik.t 

The  averaged  CSO  waveforms  for  no-move  trials  exhibited  a  series  of  peaks 
corresponding  to  the  ‘late  positive  complex'  in  the  ERPs.  The  most  robust 
peak  oocurred  at  313  msec  for  visual  no-move  trials  (P313),  and  at  281  msec 
for  auditory  (P281).  The  positive  peek  of  the  visual  P313  occurred  at  all 
midline  sites,  extending  lateraly  et  precentral  sites  and  to  lateral 
central  electrodes  immediately  adjacent  to  the  midlinei  it  was  largest  at 
aCz,  aCI  and  aC2  (Figures  16a  and  21a).  A  cor  respond i ng  negative  peak  was 
observed  only  at  peripheral  electrodes.  The  auditory  P281  was  similar  in 
topography  (Figures  16b  and  23a),  but  with  amplitudes  approximately  twice 
as  large  at  the  site  of  maximum  positivity  (aCz),  and  substantially  larger 
at  Fz,  Cz  and  aPz.  This  may  have  been  due  to  the  relative  infrequency  of 
the  auditory  stimuli  in  a  paradigm  using  visual  cues  and  feedback,  Perhaps 
due  to  this  enhancement  of  P281,  a  cor r espond i ng  negative  peak  was  visible 
at  eleotrodes  situated  just  within  the  peripheral  ring. 

In  the  visual  no-move  waveform  a  late  peak  occurred  between  470  and  525 
mseo  (P 500).  Its  topography  was  sharply  localized  with  a  pos i t i ve  peak 
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only  at  midline  cites  posterior  to  Cz  (maximal  at  aPz ) .  and  a  corresponding 
negative  peak  only  at  midline  frontal .  precentral ,  and  immediately  adjacent 
lateral  sites  (Figures  16a  and  21a) .  In  the  auditory  no-move  waveform  the 
late  wave  (P580)  peaked  between  480  and  600  msec.  A  distinct  positive  peak 
occurred  only  at  midline  central  and  anterior  parietal  sites  (maximum  at 
aPz).  while  the  cor  respond i ng  negative  peak  was  more  widespread,  occur  mg 
at  Fz.  FI  and  F2*  and  at  aCI  and  aC2  (Figure  16b  and  23a).  At  the  anterior 
central  midline  (aCz)  two  distinct  peaks  were  visible,  a  negative  peak  at 
480  msec  and  a  positive  peak  at  600  msec.  The  auditory  PS80  differed  from 
the  visual  PS00  in  that  its  negative  component  was  distinct  and  large  in 
amplitude  at  most  lateral  and  peripheral  frontal  and  central  sites. 

3.  Response  Potentials 

In  response-regi stared  averages,  mean  EMG  onset  was  65  msec  before  response 
movement  onset  in  the  visual  move  trials  and  90  msec  before  response  in 
auditory  move  trials.  Since  the  auditory  and  visual  waveforms  were 
similar,  only  the  auditory  waveforms  will  be  discussed.  The  topography  of 
movement -related  potentials  was  similar  to  the  right-handed  movement 
potentials  of  our  current  bimanual  study  (see  Section  V.0.1).  The 
movement -related  potentials  consisted  of  a  slow  ramp-like  shift  (RP). 
commencing  about  360  msec  before  response  onset,  and  a  peak- 1  ike  wave 
(NSS).  commencing  at  about  the  time  of  average  EMG  onset  and  peaking  S3 
msec  after  response  onset  (Figure  6).  These  two  components  were  localized 
to  a  few  electrodes*  and  had  slightly  different  topographies  (Figure  IS). 
Also.  the  morphology  of  the  whole  movement -related  waveform  exhibited 
distinct  differences  at  various  electrodes.  The  RP  consisted  of  a 
left- lateral ized  negative  shift,  strongest  at  Cl  and  aCI.  and  visible  also 
at  Cz.  aCz  and  aC2»  and  a  cor  responding  positive  wave  occur ing  in  two 
distinct  regional  one  left-sided  (aP3.  P3  and  aOI)  and  the  other 
right-sided  (C4.  aP2 .  aP6  and  P4).  The  N53  peak  exhibited  a  robust 
negative  component  similar  in  topography  to  the  negative  RP  component,  but 
extending  also  to  midline  and  left  frontal  sites  and  totally  absent  at  Cl. 
The  positive  component  of  NSS  was  present  only  at  I ef t -hemi sphere  sites 
(C5»  aP3 .  PI.  and  C3).  and  was  largest  at  C3.  Unlike  the  RP*  the  positive 
component  of  N55  was  absent  from  the  right  hemisphere.  The  topography  of 
NSS  (see  Figure  IS)  may  be  tentatively  interpreted  as  reflecting  the 
activity  of  the  right-hand  region  of  the  motor  cortex,  which  could 
conceivably  produce  a  current  source  strongest  at  aCI  and  aCz.  and  a 
corresponding  current  sink  strongest  at  C3.  The  bifurcated  positive 
topography  of  the  positive  RP  component  is  puzzling,  as  is  the  lack  of  any 
RP  shift  preceding  the  positive  component  of  NSS  in  several  electrodes  (C3» 
aPI  and  aP3).  and  the  absence  of  the  N5S  peak  at  Cl.  where  the  RP  negative 
shift  is  strong.  These  findings  suggest  the  activity  of  multiple 
generators,  an  issue  which  is  currently  being  examined  in  a  study  of  right 
and  left-handed  response  movements. 

C*  JLDJ-I_Lli  Analysis 

The  results  of  initial  NCP  Analysis  of  this  data  are  discussed  in  Section  V 
below. 


IV. 


THE  AO  I EEG  I  I  I  ANALYS I S  SYSTEM  (Development  also  sponsored  by  the 
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Offiee  of  Naval  Raaaaroh,  tha  National  Scianoa  Foundation,  and  tha  USAF 
School  of  Aarospaca  Medicine), 

A.  Overview  si  The  Naw  System 

A  third  generation  system,  AOIEEG  ill,  has  baan  davalopad  to  perform 
mu  1 1 i -stepped  sequences  of  analysis  on  large  amounts  of  data  (Figure  7), 

It  has  tha  oapaoity  to  handle  64  channels  of  BP  data,  plus  6  channels  of 
other  physiological  information.  Over  a  dozen  simultaneous  users  are 
handled  for  data  collection,  analysis  and  program  development.  Over  12 
man-years  of  senior  programmer  time  were  expended  over  the  past  three  years 
to  implement  and  test  this  system,  which  may  be  one  of  the  most 
oomprehens i ve  systems  currently  being  used  for  brain  potential  research. 

The  hardware  consists  of  similarly  configured  PDP11-60  and  PDP11-45 
computers,  with  BOO  and  400  MB  fixed  disks  respectively,  extended  memory 
busses  and  1,23  MB  main  memory  each.  Other  peripheral  devices  include 
9-track  tape  drives,  terminals  and  modems,  graphics  devices  and  printers, 
and  an  80-channei  A/0  converter.  The  RSX  multiprogramming  operating  system 
is  used.  The  following  functions  are  performed*  (1)  experiment  control, 
with  on-line  selection  of  stimuli  and  task-difficulty  according  to  the 
par t io ipant ' s  performance,  and  digitization  of  up  to  70  channels  at  2S6 
samples /sec;  (2)  f i ni te- impu I se- response  digital  filtering  for  selected 

passbandsi  (3)  eye-movement  contaminant  filtering  (see  Section  V,E,1); 

(4)  sorting  and  pruning  of  data  sets  by  up  to  30  stimulus,  response, 
performance  and  other  variables  so  that  experimental  conditions  do  not 
differ  in  irrelevant  faotors  (see  Seotion  IV, Bit  (5 >  computation  of 
current-source  densities  (CSOs)  from  the  scalp  potential  distributions  (see 
Seotion  V,0)|  (6)  computation  of  averaged  ERP  and  CSD  waveforms,  and 

interactive  programs  for  determining  the  latencies,  calibrated  amplitudes 
and  integrated  energies  of  visually  seleoted  peaks;  (7)  computation  of 
cross-cor relat ions,  power  speotra,  and  Wigner  Distributions  (combined 
t ime- f requenoy  domain)  (see  Section  V,B,2);  (8)  statistical  and  heuristic 

primary  feature  extraction  and  wi thin-subject  normal i zat ion  of  data  (see 
Section  V.A);  (9)  univariate  and  linear  multivariate  statistics  using  the 

BMD  package;  (10)  nonlinear,  adapt i ve- I ayered-networ k ,  multivariate 
pattern  recognition  which  determines  invariant  features  in  sets  of 
single-trial  data;  and  (11)  time-varying  ,  multichannel  linear  models  of 
event-related  signals.  All  stages  of  data  management,  signal  processing 
and  analysis  are  documented  in  printed  or  graphic  form.  Finally,  a 
menu-oriented  ’analysis  operating  system*  has  been  implemented  to  automate 
the  execution  of  the  many  subprograms  handling  data  sets  of  up  to  500  MB, 
greatly  reduoing  the  time  required  for  each  complete  analysis, 

B,  U3Jl  AD  I  SORT  Program  ±si  Formation  «n<j  fig  I  IQS  j  nfl  fil  Sets 

Special  attention  is  focused  on  the  preparation  of  data  sets,  since  we  feel 
that  this  is  a  highly  negleoted  aspect  of  experimental  control.  In  order 
to  form  balanced  pairs  of  data  sets  for  each  hypothesis,  the  total  set  of 
ar t i faot-f ree  trials  from  each  recording  is  submitted  to  the  ADISORT 
program,  This  is  an  interactive  program  which  displays  the  means,  t-tests, 
and  histograms  of  the  distributions  of  about  30  variables  for  pairs  of  data 
sets  selected  from  the  total  set  of  edited  trials,  Pmir*  of  data  sets  can 
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b«  quiokly  inspected  for  significant  differences  in  thoao  variables*  and 
prunad  of  outliar  trials  until  thay  ara  balenced*  usually  to  an  alpha  of 
•  2«  Tha  AO  I  SORT  program  faoilitatas  tha  handling  of  largo  amounts  of  data( 
and  parmits  a  high  dagraa  of  oontrol  of  var iablas  unralatad  to  tha 
partioular  hypothesis*  inoluding  tha  af foots  of  aya  movements.  It  outputs 
labalad  lists  of  trials  which  ara  than  submittad  to  tha  signal  processing 
and  analysis  programs  of  tha  ADIEE6  system. 

\ 

For  our  currant  bimodal  and  bimanual  studies*  tha  available  variables 
include  modality  and  magnitude  of  stimulus  number*  responding  hand* 
response  onset  tisie*  response  movement  force*  velocity*  acceleration  and 
duration*  response  outcome  (‘win*  or  *  lose* I*  error  from  target  pressure* 
adaptive  error  tolerance  measure  (current  performance  level)*  and  order  of 
the  trial  within  the  recording  session.  Also  included  are  indices  of  eye 
movement «  EM8  activity  and  arousal*  which  are  meesures  of  integrated  energy 
in  the  vertical  and  horizontal  EOG  channels*  the  EMG  channel  and  the  Pz 
electrode*  respectively*  computed  in  separate  900-msec  intervals  before  and 
after  the  onset  of  cue*  stimulus*  response  and  feedback. 

In  addition  to  inspection  of  experimental  variables  and  formation  of 
balanced  data  sets*  the  ADISORT  program  can  be  used  to  form  data  sets 
aooording  to  a  poster ior i  hypotheses*  suoh  as  low  vs.  high  error*  or  short 
vs*  long  response  times.  It  allows  selection  of  sets  of  trials  according 
to  a  particular  criterion*  suoh  as  a  narrow  range  of  reaction  times. 
Figure  8  illustrates  the  improved  definition  of  movement  -  re l ated  potentials 
in  a  stimulus-registered  averaged  waveform  eomputed  from  a  data  set  that 
was  prunmd  to  a  narrow  range  of  reaction  times. 


V.  DEVELOPMENT  MB  .TRIAL  APPLICATION  BE  NEU  ANALYSES  (Development  also 
sponsored  by  the  Office  of  Naval  Researoh*  the  National  Science  Foundation* 
and  the  U8AF  Sohool  of  Aerospace  Medio ine) 

A.  AnilYlil  a±  Sets  £i  Sinai  1  Trials 
1  •  filfiid  Sawn  inn  Measure 

Expansion  of  the  number  of  ohannels  recorded  greatly  improves  spatial 
resolution.  However*  the  increased  volume  of  data  (up  to  100  megabytes  per 
person)  necessitates  a  means  of  selecting  important  channels*  intervals  and 
if to  reduoe  the  amount  of  data  prior  to  final  single-trial  analysis  of 
spatial  interdependenoies.  Analysis  of  single-trial  data  sets  suggests 
that  some  trials  lack  clear  task-related  signals  (Gavins*  et  al «  1984)*  an 
observation  consistent  with  visual  observations  (Figure  9).  A  means  for 
determining  trials*  channels  and  intervals  with  significant  signals  was 
needed.  Multiohannel  features  such  as  correlation  are  useful  tools  for 
measuring  relations  between  signals*  but  the  combinator ial  explosion  of 
ohannel  combinations  makes  suoh  measures  oumbersome  for  scanning  large  data 
sets.  Single  ohannel  measures  are  preferable  beoause  of  their  low 
computational  requirements.  This  led  to  the  development  of  a  'scanning* 
method  to  eeleot  optimal  subsets  of  signal -bear ing  trials*  ohannels  and 
intervals*  and  to  improve  SNR  for  oor relat ion-based  measures.  This  method 
uses  a  single-ohannel  measure  of  filtered  amplitude  as  a  feature  for 
psttsrn  recognition.  Trials  without  consistent  task-related  signals  are 


taskj  iubj«ot  SS02 ) •  Th*  pattern  recognition 

mad  in  fifteen  overlapping  80 -msec  wide  windows  in 
from  266  to  449  msec,  which  spanned  the  P368  peak 
between- task  difference  in  ERP  amplitude.  The  P* 
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rejected  if  the  pattern  recognition  algorithm  is  unable  to  discriminate 
them  from  sets  of  'basel ine"  data  known  to  have  a  low  event  *  re  I ated  signal 
(see  Section  V.A.2.b), 

A  reasonable  criterion  for  a  scanning  method  for  single  trials  is  that  it 
yield  high  c lass i f i cat i on  scores  in  channels  and  intervals  which  have 
significant  peaks  in  the  average  waveforms.  The  obvious  candidate  for  a 
single-channel  measure  is  the  time  series  values  themselves.  An  initial 
study  was  performed  using  visuospatial  task  data  from  a  single  person 
(SS02),  Segments  of  the  time  series  80,  120  and  180  msec  in  length  (128  Hz 
sampling  rate)  were  used  as  analysis  windows.  Using  the  negative  log  of 
the  significance  of  between-task  (move  vs,  no-move  visuospatial  task) 
discrimination  as  the  output  variable,  the  progression  of  d i sor imi nab i I i ty 
over  time  was  observed  for  each  channel  by  advancing  the  analysis  window 
one  data-point  at  a  time.  In  the  P3a  interval  (266  to  445  msec),  the  PZ 
channel  had  a  robust  result,  consistent  with  the  average  ERP  (Fig,  10), 
The  same  measure  was  applied  to  the  data  from  participant  AV07  performing 
the  aud i tor y / v i sual  numeric  judgment  task  over  a  wider  range  of  latencies. 
The  results  are  shown  in  Figure  12  in  topographic  'head'  format,  (See 
Figure  11  for  explanat'on  of  the  topographic  head  displays  used  for  the 
figures  in  this  section,)  Significant  but  weak  between-task  differences 
occurred  primarily  in  the  intervals  centered  at  250,  313,  and  375  msec, 
reflecting  activity  related  to  the  N242  and  P313  peaks  elicited  by  the 
infrequent  no-move  stimuli. 

The  time  series  was  then  lowpass  filtered  at  7  Hz  to  remove  alpha  and  beta 
components.  The  filter  was  a  17  point  I  inear -phase  FIR  design  with  20  dB 
attenuation  ( , 0 1  power  gain)  at  16  Hz,  so  that  decimation  by  4  produced 
only  minor  aliasing  error.  This  produced  results  with  better  between-task 
discrimination  (up  to  about  80%)  (Figure  13), 


2,  Add  I icat ion  q±  Scanning  Measure  lo  Vi thin-Task  Analysis 

a.  p_r t,S.m  ln^t.r.Yl.1.  Basel  ine 

»>  Stimulus-Registered  A./V  Moyt  Tasks 

The  purpose  of  the  soanning  measure  was  to  choose  trials  with  a  clearer 
task-related  signal,  The  'reference'  ensemble  of  features  should  ideally 
be  derived  from  intervals  with  no  task-related  signal  so  that 
signal -bear ing  trials  can  be  optimally  discriminated.  In  the  bimodal 
experiment,  the  averaged  CSOs  show  a  relatively  small  amount  of  energy  in 
most  ohannels  prior  to  the  cue,  A  series  of  NCP  Analyses  was  performed  on 
the  visual  and  auditory  move  (correctly  cued)  trials,  using  a  time  interval 
centered  375  mseo  before  the  cue  as  reference.  For  visual  stimuli  (Figure 
14a),  the  analysis  interval  centered  at  117  msec  shows  a  pattern  of 
significant  signals  related  to  the  N117  peak  at  posterior  sites.  Some 
activity  can  also  be  seen  at  occipital  sites  in  the  adjacent  intervals  due 
to  overlap  in  the  125-msec  wide  analysis  intervals  centered  on  these 
closely  spaced  peaks,  The  anatomic  distribution  of  significant  signals  in 
the  P2,  N2  and  P3  intervals  corresponds  only  approximately  with  the 
averaged  CSO  topographies,  This  may  be  due  to  the  use  of  the  pre-cue  EEG 
as  reference,  which  is  certainly  not  free  of  task-related  activity.  In  the 
slow  wave  (SW)  interval,  the  pattern  of  significant  signal  sites  is 
separated  into  anterior  and  posterior  regions,  as  might  be  expected  for  the 
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^!®uri  .I1*  Topographic  *haad*  displays  for  NCP  Analysis  rasults  and 
J|,,tr,but,on»*  Thirty-aight  ohannals  art  shown;  tha  11 
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sign  fieanoa  soala  is  at  tha  top  of  Figuras  12-16,  whila  tha  CSO  soala 
is  givan  baiow  in  Figuras  14-16. 
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Figure  12*  Single  channel  scanning  measure  using  data-points  from 
unfiltered  single-trial  time  series  as  features  for  pattern 
recognition*  Discrimination  of  oorreotly  cued  (move)  vs*  misoued 
I  no -move)  AV07  visual  trials.  Analysis  windows  120-mseo  wide  were 
centered  at  IS  latencies  from  250  mseo  before  to  8SS  msec  after 
stisMilus  onset*  Eleotrode  sites  with  higher  signifioanoe  levels  are 
indicated  by  higher  density  of  dots*  (See  Figure  11  for  explanation 
of  ‘head*  diagrams). 
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AUDITORY  MOVE  VS.  NO-MOVE,  FILTERED 


Figura  13,  8 ingle- channel  soanning  measura  using  filtered  data, 

Thraa  data  points  froai  singla-trial  t iwa  sarias  lowpass  filtarad  at  7 
Hx  war a  usad  as  faaturas  for  between- task  pat  tarn  raoognition. 
Oiser  Istinat  ion  of  oorraotly  ouad  (nova)  vs,  misoued  (no-siovel  trials, 
in  saparata  analyses  of  visual  (top)  and  auditory  (bottom)  conditions, 
Signifioanoa  lavals  of  disor  isiinat  ion  ara  highar  and  isora  anatosii oal  I y 
distinct,  aspaoialiy  in  tha  N2-P3  ranga,  than  thosa  achiavad  using 
unfiltarad  data  (Figura  12),  Filtarad  time  sarias  ara  usad  in  tha 
single-ohannel  soanning  analysas  in  Figuras  14*16,  (Saa  Figura  11  for 
explanation  of  'head*  diagrams,) 
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visual  slow  wavs,  which  is  nsgativs  frontally,  pos i t i vs  posteriorly,  and 
waak  at  csntral  locations.  In  comparison,  ths  auditory  slow  wave  (SW) 
pattsrn  is  distributed  over  a  single  cent ro-par i eta  I  area  (Figure  14b).  • 

For  auditory  stimuli  (Figure  14b).  the  pattern  of  s i gn i f i can t - s t gna I  sites 
related  to  the  auditory  N109  peak  is  anterior,  strongest  at  lateral  central 
sites  (Cl.  C2.  and  C3).  with  weaker  activity  at  some  far  lateral  frontal, 
precentral  and  parietal  sites.  The  auditory  P171  pattern  is  also  mostly 
anterior.  but  is  strongly  focused  at  right-sided  lateral  central  and 
precentral  sites  adjacent  to  the  midiine.  The  P242  pattern  shows  some  * 

correspondence  to  the  CSD  distribution.  but  with  a  right-sided 
preponderance  of  significant  signal  sites. 

2)  Response  Registered  A/V  Tasks 

The  results  of  the  response- reg i s tered  analyses  were  similar  for  visual  and  • 
auditory  trials  (Figure  IS).  In  the  3  intervals  leading  to  movement  onset, 
corresponding  to  the  slow  readiness  potential  (RP)  shift,  the  significant 
sites  were  predomi nent I y  left-sided,  strongest  at  central  sites  next  to  the 
midline.  and  increasing  in  strength  with  the  EMG  onset.  In  the  interval 
centered  on  the  maximum  post -response  peak  of  the  movement  -  re  I ated 
waveforms  (N62  in  visual.  N55  in  auditory),  there  was  a  large  increase  in  • 
the  number  of  sites  and  strength  of  the  significant  signals,  cor r espond i ng 
well  in  anatomic  distribution  to  the  positive  and  negative  components  of 
these  peaks  in  the  CSO  waveforms.  Note  that  discrimination  of  activity  in 
the  EMG  channel  persists  into  the  last  interval,  centered  about  300  msec 
after  response  onset,  while  significant  cerebral  signals  are  almost  absent. 

b.  ‘Random  Pile'  Bait  I  tot 

1)  Method 

Although  the  energy  in  the  pre-cue  averaged  waveform  was  low.  it  was  not 
zero.  perhaps  due  to  task-related  preparatory  processes.  No  fixed  time 
interval  in  an  experiment  is  really  representative  of  a  'neutral'  baseline 
state.  A  statistical  technique  was  therefore  developed  to  generate  an 
ensemble  of  reference  data  which  would  approximate  a  'neutral*  baseline 
having  minimal  s t imu I  us- r eg i s t ered  activity  and  average  statistical 
properties  similar  to  each  channel  of  recorded  data.  Simply  put.  this 
technique  averages  task-related  signal  statistics  over  the  whole  trial  by 
synthesizing  a  locally  stationary  process  for  each  channel  from  randomly 
chosen  epochs  of  recorded  EEG.  This  process  has  first  and  second  order 
statistics  which  should  differ  from  those  of  the  underlying  baseline  EEG 
only  by  small  t ime- i ndependent  constants.  To  compile  the  set  of 
pseudo-base  I i ne  data,  registration  points  are  chosen  at  random  within  the 
recording  epoch.  For  each  channel,  the  features  are  the  filtered  and 
decimated  time  series  values  in  the  intervals  immediately  following  the 
registration  points.  The  ensemble  of  such  intervals  over  all  task 
conditions  serves  as  the  reference  class.  The  data  set  representing  the 
task-related  signals  is  composed  of  intervals  centered  on  the  latency  of  a 
particular  peak  of  the  averaged  waveform. 

Implicit  to  this  idea  of  a  randomized  data  reference  is  an  assumed  model. 
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Figure  14a.  Within-task  single-channel  scanning  matura  of  visual 
aiova  trials  using  pre-oue  t  i  me  sarias  as  rafaranoa  data  (top)* 
ooaiparad  with  tha  avaragad  our  rant  souroa  dansity  (C8D)  distributions* 
at  tha  oantarpoint  latanoy  of  aaoh  NCP  Analysis  intarval  (bottom). 
NCP  Analysis  intarvals  wars  12S  msao  wide*  exoept  for  tha  prastimulus 
and  Stf  apoohs  which  wara  290  msao  wida.  Channals  without  significant 
task -related  signals  do  not  appear  in  tha  NCP  results.  (Sea  Section 
V.A»2.a.1  of  text  for  disoussion*  and  Figure  (I  for  explanation  of 
"head*  diagrams.) 
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where  x(t)  is  the  observed  process,  n(t)  is  a  wi de-sense  stationary  colored 
no • *•  process,  and  s(t)  is  a  non*stat ionary  signal  process.  For 
simplicity,  we  assume  an  additive  linear  model.  However,  very  little  is 
assumed  about  the  character i st i cs  of  n<t!  and  s(t),  and  non-stat lonar i ty  is 
permitted.  In  practice,  these  two  processes  may  be  very  similar  and  in 
fact  may  well  have  nearly  identical  power  spectra  below  8  Hz,  The 
fundamental  difference  between  them  is  the  non-stat ionar i ty  of  the  signal, 
which  is  time-locked  to  experimental  events.  The  influence  of  this 
non-stat i onar i ty  on  the  sum  can  be  greatly  reduced  by  the  introduction  of  a 
random  variable  which  is  uniformly  distributed  between  0  and  T-w,  where  T 
is  the  trial  length  and  w  is  the  analysis  window  length: 

0  •''U10,  T-wl 

Then  the  first  order  statistics  of  x(t+0)  are  (for  0  <*  t  <*  wl  : 

ECx(t*©)3  *  ECn ( t  +0 ) 3  ♦  ECs  (  t  +0  )  3 

The  noise  term  is  clearly  constant  over  t,  since  n  is  stationary.  The 
second  term  is  the  expected  value  of  the  local  time  average  of  the  signal, 
and  is  also  a  constant.  Thus,  the  sum  is  constant,  and  not  a  function  of 
time.  The  second  order  statistics  are 

Etx ( 1 1  ♦  0)  x(t2  ♦  0)3  =  tS)  n(^tS)  j  f  E  d (?■*-  + 

+  E  c +u0)s(*x+e)J  i-  E[s( ti+e)hCt*+  e)^ 

For  each  of  these  terms,  the  expectations  are  nearly  identical  for  any 
choices  of  tl  and  t2  such  that  <  1 1  -  1 2 )  is  constant,  since  near  I y  all  of 
the  possible  times  will  be  equally  likely  (the  time  points  considered  may 
differ  for  as  many  as  w  points  out  of  T-w  possible  points),  For  w  ((  T, 
the  individual  time  dependencies  can  be  dropped,  yielding 

ECx(t1  ♦  0)  x(t2  ♦  0)3  »  £n*(‘t,'t3L)  +  +  trtj  * 

Therefore,  x(t  ♦  0)  is  locally  (for  0  <  *  t  <  *  w)  stationary.  Also,  if 

(0)  «  i^co ) 

and 

(6  ns  (o) ,  Co)  <  <  (o) 

(small  task-related  signal  power,  small  correlation  between  signal  and 
noise),  then 

Thus,  a  locally  stationary  process  has  been  manufactured  with  small 
constant  t ime- i ndependent  differences  in  first  and  second  order  statistics 
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from  the  theoretical  underlying  baseline  EEG  process, 

2)  AppI i cat  ion  _L£  Miscued  (No-Move)  A/_V  Data 

NCP  Analyses  using  randomi zed-data  reference  sets  produced  stronger,  more 
anatomically  distinct  results  than  analyses  using  pre-cue  data  as  reference 
(compare  top  of  Figures  16a  and  16b  with  16c),  Analyses  of  the  no-move 
trials  of  participant  AV07  for  the  infrequent  miscued  visual  (Figure  16a) 
and  auditory  (Figure  16b)  stimuli  produced  sequences  of  patterns  consisting 
of  electrode  sites  with  task  -  re  I ated  signals  strong  enough  to  allow  65-75% 
of  the  single  trials  of  the  task  to  be  discriminated  from  the  random 
baseline  data.  The  intervals  centered  on  the  N1,  P2,  N2,  P3  and  slow  wave 
( SW)  peaks  show  a  progression  of  task-related  patterns  which  overlap  in 
time.  For  miscued  visual  stimuli  (Figure  16a),  a  r i ght - I atera I i zed  pattern 
at  occipital  and  posterior  parietal  sites  is  evident  in  the  N117  interval, 
and  also  in  the  overlapping  P180  interval.  The  auditory  P109  interval 
(Figure  16b)  lacks  this  posterior  pattern;  the  significant  signals  occured 
at  midline  and  lateral  central  and  precentral  locations,  in  the 
prestimulus  interval  centered  500  msec  before  stimulus  onset  (500  msec 
after  cue),  there  is  a  posterior,  r i ght - I atera I i zed  pattern  resembling  the 
posterior  pattern  of  the  visual  N117  interval.  This  is  especially  evident 
in  the  visual  condition  (Figure  16a),  but  is  also  visible  in  the  auditory 
condition  (Figure  16b),  This  interval  spans  the  early  CNV  (*0*)  wave, 
which  is  known  to  depend  on  the  modality  (in  this  case  visual)  in  which  a 
cue  is  presented.  The  anterior  patterns  in  the  visual  N117,  P180,  N242  and 
P313  intervals  may  reflect  the  overlapping  activity  of  these  components,  as 
wall  as  the  CNV  resolution.  This  is  due  in  part  to  the  overlap  of  the 
125-msec  wide  analysis  intervals  centered  on  these  closely  spaced  peaks. 
The  visual  P180  and  auditory  P172  both  have  strong  anterior  patterns 
focused  at  Cz,  and  in  the  visual  P180  interval  strong  also  at  anterior 
central  sites  (aCI,  aCz  and  aC2),  In  the  N210  interval  for  auditory 
stimuli  the  anterior  pattern  moves  forward  to  precentral  sites,  where  it  is 
r i ght - 1 ateral i zed  and  strongest  at  aC2,  whereas  in  the  visual  N242  interval 
there  is  not  much  ohange.  In  the  auditory  P297  interval,  there  is  a  strong 
pattern  consisting  of  midline  sites  from  Fz  to  aPz,  and  lateral  precentral 
sites  (aCI  and  aC2)»  with  strong  signals  also  at  right  anterior  parietal 
locations  (aP4  and  aP6),  In  contrast,  the  visual  P313  pattern  lacks  the 
midline  sites,  and  resembles  the  auditory  P297  pattern  only  at  the  lateral 
precentral  (aCI  and  aC2)  and  anterior  parietal  (aP4  and  aP6)  sites.  This 
relative  modality  dependence  of  the  P3  amplitude  distribution  may  be  due  to 
the  much  greater  amplitude  of  the  rarer  auditory  no-move  peak,  and  to 
overlapping  effeots  from  the  modality-dependent  N2  peak,  The  slow  wave 
(SW)  interval  does  not  overlap  adjacent  analysis  intervals,  and  a  clear 
differenoe  between  visual  and  auditory  conditions  can  be  seen.  The  visual 
SW  pattern  is  divided  into  separate  anterior  and  posterior  regions,  as 
might  be  predicted  from  the  slow  wave  distribution  for  visual  stimuli.  The 
auditory  slow  wave  pattern  is  more  evenly  distributed, 


3,  Conclusion 

From  these  results,  it  seems  that  the  single-channel  scanning  measure 
achieved  its  aim  of  reducing  the  data  set  by  isolating  intervals  and 
channels  with  task-related  signals  prior  to  measurement  of  between-channe I 
timing  relations.  The  poor  temporal  resolution  of  the  scanning  measure 
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Figure  18.  Use  of  pattern  recognition  to  seleot  sets  of  individual  trials 
with  consistent  task-related  signals.  The  averaged  waveform  of  the 
original  188  trials  is  at  top.  The  middle  'purified*  waveform  is  the 
average  of  181  trials  whioh  were  oorreotly  discriminated  from  pre-oue  data 
by  the  pat  tern- reoogni t ion  algorithm  using  data  points  from  a  129  mseo 
interval  centered  on  the  P2  peak.  There  is  an  enhancement  of  the  N1.  P2 
and  N2  peaks,  as  well  as  the  pre-stimulus  and  pre-movement  slow  potentials 
even  with  34%  fewer  trials  than  the  original  average.  The  68  inoorreotly 
elassified  trials  produoe  an  average  waveform  (bottom)  whioh  laoks 
resemblance  to  the  original  average  and  is  dominated  by  ongoing  EEQ 
aot i vi ty • 
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results  from  its  epplioation  to  I owpass - f i I tered  data.  This  will  not 
affeot  results  of  the  second  pass  of  NCP  Analysis  which  will  achieve  good 
temporal  resolution  by  measuring  interchannel  timing  relations  on 
unfiltered  data. 


B.  Av.tr-lfll.tf  EvsKitf  XiOH.  Ser  ies  Analysis 
1.  ‘Pur  Iliitf  ‘  £B£J. 

A  great  deal  of  effort  has  been  expended  in  attempts  to  form  improved 
estimates  of  the  averaged  ERP  (see  reviews  in  McGi I lem,  et  al . ,  1381.  de 
Weerd  1901}  and  Qevins,  1984).  Most  methods  assume  that  task  -  re  I ated 
signals  are  present  in  every  trial  and  also  have  inherent  assumptions  about 
the  statistical  properties  of  signal  and  noise.  We  have  developed  a 
method  for  ERP  estimation  without  these  assumptions.  To  do  this,  separate 
averages  are  formed  from  trials  which  are  correctly  or  incorrectly 
classified  by  the  single-channel  scanning  pattern  recognition  procedure 
described  above  (Figure  17).  In  the  averages  of  correctly  classified 
trials,  the  ERP  peaks  are  enhanced  in  comparison  with  the  original  averages 
(Figure  18),  The  averages  of  incorrectly  classified  trials  resemble  the 
background  EEG.  Several  dozen  applications  of  this  technique  suggest  that 
it  is  successful  in  identifying  trials  with  poor  SNR,  and  provides  a  simple 
siethod  of  ERP  estimation  without  the  a  priori  assumptions  about  the 
oharaoter i st ics  of  an  unknown  signal  made  in  techniques  such  as  Wiener  or 
minimum  mean-square  error  filtering.  While  the  classifier  we  use  is  a 
nonlinear,  layered  network  algorithm,  similar  ‘purified  ERPs*  were  also 
obtained  with  s implmr  discriminant  functions  suoh  as  those  produced  by  the 
widely  available  BM0P7M  program. 


The  simplified  procedure  may  be  summarized  as  follows! 
single  trials  to  remove  alpha  and  beta  components; 
resulting  time  series  to  produce  features  for  a  simple 
disoriminant  funotion.  Choose  points  from  a  time  region 
class,  for  example  the  P3  peak.  The  other  class  can  be 
interval  with  low  average  energy  (such  as  the  pre-st imu 
the  reference  class  may  be  obtained  by  the  'random  data' 
described  in  the  previous  section;  3)  Compute  the  Fi 
funotion  from  these  classes  and  attempt  to  classify  the 
Compute  the  average  of  the  unfiltered  trials  correspondi 
trials  which  were  correctly  classified  in  (2).  This  is 
est imate. 


1  )  Filter  the 
2)  Decimate  the 
(F i sher )  I i near 
of  interest  as  one 
timepoints  from  an 
I  us ) •  Pr efer ab I y , 
baseline  technique 
sher  discriminant 
feature  sets;  4) 
ng  to  the  f i I tered 
the  enhanced  ERP 


2*  Wiflntr.  <Tt,mi‘FrtflMtngV>  Distributions 

The  ERP  waveform  is  a  function  of  time  and  does  not  provide  explicit 
frequency  information.  Power  spectra  of  ERP  waveforms  provide  frequency 
information  but  obscure  time-dependent  phenomena.  A  view  of  the  spectrum 
as  it  ohanged  over  time  would  give  a  new  view  of  the  evolution  of  different 
frequency  components  of  the  ERP.  A  simple  approach  would  be  to  compute  the 
speotrum  over  highly  overlapped  windows  of  the  average  ERP.  However,  such 
a  ‘speot rogram"  would  smear  together  events  within  each  analysis  window. 
While  there  is  an  unavoidable  trade-off  between  frequency  and  time 
resolution  (known  as  the  Heisenberg  Uncertainty  Principle),  the  spectrogram 
is  an  inflexible  way  of  determining  this  choioe.  The  octave-band  filters 
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of  do  Weerd  (1981)  at  laaat  optimally  trada  off  tima  rasolution  for 
frmqumney  ra solution  in  diffarant  obaarvad  bands,  but  tha  ohoicat  are  still 
pre-determi nad  and  inflexible,  A  preferred  method  is  to  compute  a  function 
of  tima  and  frequency  called  tha  Signer  Distribution  (Classen  and 
Meek lenbrauker ,  1980 j  Janze  and  Kaizer,  1983)  which  approximates  the 

i nstantaneous  energy  for  a  given  time  and  frequency,  Its  correspondence  to 
such  a  function  is  only  approximate,  because  it  can  have  negative  values, 
and  beoause  the  Uncertainty  Principle  still  requires  us  to  give  up  some 
frequency  resolution  for  a  narrowing  of  time  resolution,  However,  the 
Wigner  transformation  gives  a  raw  ‘distr ibut ion'  which  can  be  averaged  over 
many  different  time  and  frequency  regions  to  give  valid  energy  estimates. 
Thus,  the  Wigner  Distribution  is  a  basic  t ime- frequency  function  without 
fixed  resolution  trade-offs,  In  fact,  all  other  t ime- frequency 
distributions  of  the  general  type  called  the  Cohen  class  (Cohen,  1966), 
including  the  spectrogram,  can  be  derived  as  time-smeared  versions  of  the 
Wigner  Distribution,  in  practice,  the  ‘purified*  ERPs  show  strong  enough 
energy  'peaks*  in  the  Wigner  Distribution  that  very  simple  interpretations 
of  the  time  and  frequency  looations  of  signal  energy  are  valid.  This  is 
true  because  local  averages  will  all  show  the  predominance  of  energy  in  the 
observed  peak  regions. 

The  Wigner  Distribution  is  defined  as 

%  (+,»)  =  f  e’1"*  dt 

with  x(t)  the  average  ERP,  omega  the  frequency  variable,  and  tau  the  dummy 
integration  variable,  (Notei  in  practice,  finite  intervals  are  analyzed, 
giving  rise  to  the  windowed  or  'Pseudo*  Wigner  Distribution,  For  properly 
windowed  signals,  this  causes  a  minor  blurring  in  the  frequency  direction, 
but  no  blur  in  the  time  direction),  integration  over  time  or  frequency 
yields  the  power  speotral  density  and  the  instantaneous  power  functions 
respect i ve I y ,  i,e«,  ^ 

/*tyxCtjU>)olt  ^  ly^w))*  ari-  /  W*  (t, =  | 

-a®  J  ~.-oo 

Fur  ther , 

r  IXM) */  t 

•CD 

the  expected  value  of  the  time  variable  is  the  average  time,  which  may  be 
viewed  as  the  group  delay,  and 

-  IxC/j/4  J  00  C'fi  *0 

the  average  frequency  value  for  eaoh  time  point  is  the  i nstantaneous 
frequency. 

The  Wigner  Distribution  is  actually  best  calculated  from  the  analytic 
signal  of  the  'purified*  average,  whioh  is  the  comp  I  ex - va I ued  signal  whose 
real  part  is  the  original  time  series,  and  whose  imaginary  part  is  the 
Hilbert  Transform  of  the  real  part.  The  magnitude  of  this  complex  time 
series  is  the  temporal  envelope  of  the  real  time  series, 
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Figure  19  shows  the  (Pseudo)  Wigner  Distributions  for  several  channels  of 
the  average  of  correotly  classified  trials  for  participant  SS02  in  a 
previous  visuomotor  experiment* 

Figures  20  through  23  show  (Pseudo)  Wigner  Distributions  for  several 
channels  of  the  average  of  correctly  classified  trials  for  AV07« 


C.  Test  S±  Principal  Components  Anal  vs  i  s  (PCA)  f  o_r 
Oef  in  i  no  AnajjotLt  W)  Q.H<?wt 

Principal  Components  Analysis  (PCA)  has  become  a  standard  procedure  for 
primary  feature  extraction  in  ERP  analysis*  Recently*  this  procedure  has 
come  under  criticism  from  a  variety  of  quarters  (Bevins*  1980;  Rosier  and 
Manzey*  1981;  Hunt*  1984;  Wood  and  McCarthy*  1984;  Bevins*  1984). 
Questions  have  been  raised  concerning  its  assumptions  of  normal  data 
distributions*  orthogonality  of  variance  from  different  physiological 
generators*  and  invariance  of  ERPs  across  trials*  as  well  as  the 
appropr i ateness  of  rotating  factors.  Studies  using  synthesized  waveforms 
have  indicated  misal locat ion  of  variance  among  factors  and  subopt imal 
feature  extraction  (Wood  and  McCarthy*  1984).  Nevertheless*  we  tested  the 
usefulness  of  PCA  for  determining  the  time  intervals  for  single-trial 
be tween -channel  correlation  analysis.  Our  previous  method  was  to  center 
analysis  windows  of  a  standard  size  on  ERP  peak  latencies  for  each 
participant  as  assessed  by  an  expert  from  the  averaged  ERPs.  An  effective 
automated  method  of  interval  seleotion  could  possibly  provide  greater 
preoision  and  objectivity*  especially  in  the  case  of  overlapping 
'components'.  To  this  end*  a  series  of  PCAs  with  Var imax  rotations  were 
performed  on  subaverages  of  'move*  and  'no  move'  trials  from  the  data  of 
one  to  four  partioipants  in  a  previous  study  (Bevins*  et  al*  1983*  1984). 
Averaged  digitized  values  were  lowpass  filtered  (McC I e I  I an-Par ks  filter 
with  outoff  at  approx.  30  Hz)  and  decimated  by  2  to  reduce  the  number  of 
variables  in  the  PCA.  Analysis  windows  were  then  set  according  to  the 
latenoies  where  the  significant  factor  loadings  for  'large*  factors  (i.e. 
those  representing  the  most  variance)  were  consistently  large  over  a 
continuous  time  interval.  Ten  to  thirteen  factors  were  required  to  account 
for  more  than  90%  of  the  variance;  of  these*  some  were  similar  in  latency 
and  duration  to  peaks  in  the  ERP  waveforms*  while  others  were  not. 
Further*  the  amount  of  variance  accounted  for  by  factors  corresponding  to 
the  peaks  which  were  maximally  different  between  conditions  were  not  always 
correspondingly  large.  None  of  the  PCA-determi ned  windows  performed  as 
well  in  between-task  pattern  recognition  analysis  using  single-trial 
correlations  as  the  180-msec  wide  windows  centered  on  the  manua I  I y -p i eked 
ERP  peaks.  It  is  difficult  to  determine  which  of  the  possible  problems 

with  PCAs  led  to  its  inferior  performance  as  a  means  of  determining 
intervals  for  pattern  recognition  analysis  of  correlations.  The  conclusion 
we  draw  is  that  use  of  windows  centered  on  the  manually  picked  ERP  peaks  is 
superior  for  our  purposes. 


o.  current  SsuLst  P«n»*JY  I rant  form 


1.  Derivation  of  Current  Source  Dens i tie* 


VISUAL  MOVE  TASK 
AV07 


Figure  2«a.  Twelve  representative  ohannele  of  current  seuree  densities  for 
visual  neve  task  from  participant  AVOT.  Average  of  trials  (22  to  M 
depending  on  the  ehanneli  which  had  task 'rotated  signals  in  the  P2  and  PS 
intervals  and  whieh  were  further  restricted  to  a  reaction  tiae  of  S2S  sseo 


VISUAL  MOVE  TASK  Z—  Wiflner  Distribution 

Uf  '*  I  h  *  I.  I  I 
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Current  Density 


Figure  21a.  C80s  for  infraquant  visual  no -mo vs  task  from  participant  AV07 
Avaraga  of  trials  (21  to  90  depending  on  the  channel)  which  had 
task-related  signals  in  the  P2  and  P3  intervals. 


Current  Density 


Figure  22a.  CSOs  for  auditory  move  task  from  participant  AV07.  Average  of 
25  to  35  trials  depending  on  the  channel  which  had  task-related  signals  in 
the  P2  and  P3  intervals  and  which  were  further  restricted  to  a  reaction 
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Average:  39 
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In  order  to  sharpen  the  spatial  localization  of  brain  activity  recorded  at 
the  scalp,  we  convert  the  event  -  re  I ated  potential  time  series  to  current 
source  densities  (CSO),  This  method  computes  the  curvature  of  the 
potential  distribution  (the  Laplacian  transform)  at  each  electrode  site. 
This  reflects  the  sources  and  sinks  in  the  flow  of  electrical  current  at 
the  scalp,  which  presumably  reflects  the  emergence  and  re-entry  of  currents 
through  the  skull  (Hjorth,  1975,  1982),  The  advantages  are  that:  1)  the 
CSO  is  independent  of  the  choice  of  reference  electrode  site;  2)  the 
activity  of  current  sources  outside  the  ring  of  electrodes  immediately 
adjacent  to  a  particular  site  are  suppressed,  and  as  a  result;  3)  the 
crosscor re  I  at i on  between  scalp  electrodes  due  to  volume  conduction  is 
reduced.  When  applied  to  ongoing  EEG,  it  was  reported  that  the 

interelectrode  correlation  is  reduced  by  a  factor  of  2  to  4,  depending  on 
the  distance  between  electrodes  (Hjorth,  1975),  The  ability  of  this  type 
of  transform  to  provide  sharper  localization  of  sources  of  abnormal 
electrical  activity  has  been  demonstrated  in  several  clinical  EEG  studies 
(Spehr,  1976;  Hjorth,  1982), 

We  are  using  CSO  derivations  in  two  studies  of  evoked  electrical  time 
series:  a  visuomotor  numerical  judgment  study  comparing  right-  and 

left-handed  response  movements,  and  the  bimodal  (audi tory/v i sual )  numerical 
judgment  study  described  in  this  report.  The  former  was  recorded  with  a 
26-electrode  montage  and  the  latter  with  a  49-electrode  montage.  After 
applying  the  CSO  transform  to  each  single  trial,  averaged  event  -  re  I ated 
CSDs  were  computed,  and  the  latencies  of  the  major  peaks  determined 
(Section  III).  Then  the  sets  of  single-trial  data  were  submitted  to  the 
first  scanning  stage  of  NCP  Analysis  (as  described  above)  in  intervals 
centered  on  these  peak  latencies. 

We  have  implemented  the  CSO  transform  in  software,  using  measures  of 
interelectrode  distances  determined  in  two  ways:  reconstruction  of  the 
three-dimens ional  positions  of  the  electrodes  using  a  stereo 
photogrammet r i c  system  (see  next  section),  and  direct  measurement  of 
interelectrode  distances  with  a  flexible  ruler. 

Since  the  electrical  activity  at  scalp  locations  outside  the  area  covered 
by  the  electrode  montage  is  not  available,  CSDs  calculated  for  peripheral 
electrodes  of  the  montage  have  fewer  'correction*  terms,  and  the  validity 
of  their  amplitude  is  questionable.  These  electrodes  are  excluded  from 
further  analysis. 

Averaged  event-related  CSOs  have  a  higher  degree  of  spatial  resolution  than 
corresponding  referential  ERPs  (Thickbroom,  et  al ,  1984).  For  example,  in 
movement - reg i stered  ERPs  for  finger  movements,  the  readiness  potential  (RP) 
and  post -movement  positivity  (PMP)  are  widespread  over  frontal  and  central 
sites,  with  larger  amplitudes  at  central  sites  contralateral  to  the 
responding  hand  (Figs,  24  and  25),  in  the  corresponding  averaged  CSOs 
these  components  are  more  localized.  The  RP  and  PMP  exhibit  distinct  phase 
reversals  between  the  anterior  midline  central  electrode  (aCz)  and  just  one 
or  two  lateral  electrodes  opposite  to  the  responding  hand  (C3  and  aPI  for 
right  hand  movements;  C4  for  left  hand  movements).  This  may  be  an 
indication  of  a  localized  current  source  midway  between  these  sites,  near 
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Figure  23.  MovMtnt*rtfiit«r«d  ERPs  for  on  isometrio  loft  index  finger 
oont root  ion.  Avorago-of -por iphoral -oiootrodos  roforonoa  (upper) ,  and 

ourrent  eouroe  density  (lower).  C80  has  improved  localization  of 
contralateral  movesient* related  aotivity.  Compare  with  Figure  24. 
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the  oortioal  motor  area  for  hand  and  finger  movements.  The  CSD  time  series 
for  central  sites  ipsi lateral  to  the  responding  hand  are  quite  flat  in  the 
recordings  examined  to  date.  Such  an  i nterpretat i on  is  in  agreement  with 
measurements  of  event  -  re lated  magnetic  fields.  Similarly,  for  both 
auditory  and  visual  stimuli,  the  primary  sensory  peaks  ( N 1 0  0  >  are  more 
localized  in  the  CSDs  than  in  cor r espond i ng  ERPs  (see  Figures  4  and  5). 

2,  Petermi n i no  Electrode  Posi t ions 

In  order  to  determine  the  precise  positions  of  the  electrodes  for  current 
source  density  transform  and  spatial  deconvolution,  we  used  a  system  of 
stereophotogrammetry  developed  for  orthodontic  use,  A  series  of  stereo 
pictures  of  the  par t i c i pan t ' s  head  with  electrode  cap  in  place  was  taken. 
The  electrode  positions  in  the  photographs  were  digitized  on  a  digitizing 
table,  along  with  fiducial  marks  added  by  the  camera,  Computer  software 
then  reconstructed  the  three-dimensional  positions  of  all  electrodes.  This 
was  done  for  2  par t i c i pants .  For  one  person,  before  and  after  measurements 
were  made  to  investigate  the  stability  of  electrode  positions  across  the 
recording  session.  The  cap  was  found  to  be  riding  up  at  the  back  of  the 
head  by  as  much  as  17  millimeters.  In  subsequent  recordings  we  have 
amelioriated  this  problem  by  allowing  the  cap  to  settle  before  applying 
electrodes.  The  measures  of  electrode  distances  before  and  after  the 
recording  session  were  used  to  calculate  current  source  density  transforms 
to  assess  the  dependence  of  CSOs  upon  variations  in  measurements.  These 
CSDs  differed  only  slightly,  although  individual  interelectrode  distances 
differed  by  as  much  as  10%.  We  are  currently  determining  electrode 
distances  for  CSD  transforms  by  measuring  pairwise  i n *  ere  I ect rode  distances 
with  a  flexible  plastic  ruler.  The  standard  deviations  of  these  distances 
across  persons  were  under  3  mm,  averaging  about  2  mm.  This  consistency 
suggests  that  a  standard  set  of  interelectrode  distances  may  be  used, 
especially  since  the  sensitivity  of  the  CSD  transform  to  small  variations 
in  measurement  is  slight. 


E,  Arti  fact  PUfffUPO  MJOSl  EJ  UH-LPfl 

i .  £.it  MP-vmim  Cgntynintpsi 

The  general  problem  of  artifact  filtering  is  more  difficult  than  that  of 
artifact  detection.  Filters  of  various  types  have  been  developed  to  remove 
artifacts  from  brain  signals,  with  the  most  effort  directed  towards 
removing  eye  movement  potentials  (EOG),  Although  Johnson  and  associates 
( Johnson ,  MX  All  1979)  have  made  a  good  initial  start  using  optimal  digital 
filters,  the  removal  of  muscle  potential  contaminants  (EMG)  remains 
unsolved.  Current  techniques  for  removing  eye  movement  artifacts  rely  on 
subtraction  of  a  weighted,  simultaneously  recorded  EOG  signal  from  each  EEG 
channel,  an  idea  first  put  into  practice  by  Girton  and  Kamiya  (1973),  and 
reoently  improved  by  several  I aborator i es ,  Ver lager,  et  el.  (1982)  present 
the  following  method,  typical  of  most  current  filtersi  record  an  EOG 
channel  from  electrodes  above  and  below  one  eye,  determine  mathematically 
what  an  appropriate  'scaling  factor*  should  be  for  each  E EG  channel,  and 
then  subtract  the  EOG  signal  multiplied  by  its  cor  respond i ng  scaling  factor 
from  eaoh  EEG  signal |  repeat  the  procedure  using  an  EOG  electrode  placed  at 
the  outer  canthi  for  horizontal  eye  movements.  The  method  most  commonly 
used  to  determine  the  soal ing  factor  tends  to  reduce  the  amplitude  of  the 


gram  of  spectral  subtraction  method  of  eye-movement 
;ee  discussion  in  Section 
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EEQ  signal  as  wall  as  ramova  tha  contaminating  aya-movamant  signals* 
Qratton  Ml  a  I .  (1983)  attampt  to  ramady  this  problam  by  ramoving  an 

astimata  of  tha  undarlying  avant*ralatad  EEQ  and  EOQ  activity  prior  to 
calculation  of  tha  scaling  faotors*  Fortgans  and  da  Bruin  (1983)  racord 
four  (obliqua)  EOQ  derivations*  and  calcuiata  four  scala  factors  to  obtain 
battar  astimatas  of  var tical  and  horizontal  EOQ  contaminants*  This  appaars 
to  offar  improvad  rasults*  particularly  in  frontal  and  prafrontal  channels. 

While  these  algorithms  represent  good  first  approximation  aya-movamant 
artifact  filters*  a  number  of  improvements  remain  to  be  made*  In 
particular*  tha  transmission  of  EOQ  potentials  along  tha  scalp  and  through 
intervening  tissues  would  seam  to  be  frequency  dependant*  thus  requiring 
different  scaling  factors  for  different  frequencies*  Although  this  was 
first  suggested  by  Whitton*  at  a  I  *  (  19781*  it  was  only  recently  that 
'short-time*  spectral  subtraction  algorithms  ware  fully  applied  to  this 
problam  (Bonham*  Ml  a  I »  In  Prep*)*  In  this  method*  differences  in  the 
spectral  signatures  of  eye  movement  and  brain  signals  are  used  to  create 
filters  which  remove  eye  movement  contaminants  from  each  trial  (Fig*  26)* 
Estimates  are  computed  for  the  spectral  density  of  the  vertical  and 
horizontal  eye  movement  channels*  and  the  cross-spectral  density  of  the  eye 
movement  channels  with  each  of  the  brain  signal  channels*  Convolving  the 
inverse  Fourier  transform  of  the  ratio  of  these  two  quantities  with  each 
trial  of  eye  movement  channels  results  in  estimates  of  the  eye  movement 
contamination  of  the  brain  signal  for  that  trial*  This  estimate  is 
subtracted  from  each  trial  of  each  brain  signal  channel*  The  system 
transfer  function  estimate  is  updated  periodically  by  computing  new  values 
for  the  spectral  and  cross-spectral  densities  from  recent  trials*  In 
initial  evaluations  the  method  appears  to  work  very -well  (Fig.  27)* 
exceeding  the  performance  of  the  Qratton  et  al •  (1983)  amplitude 
subtraction  method  (Fig.  28)  as  assessed  by  a  decreased  ability  to 
distinguish  unartifaoted  from  filtered  trials  using  a  nonlinear  classf ier. 

2*  >  UMLMkMMM. 

«•  Overview 

The  current  methods  for  editing  polygraphs  rely  on  the  off-line  procedure 
of  detecting  artifacts  by  visual  inspection*  Such  a  procedure  is  very 
labor-intensive*  This  database  is  a  first  step  in  the  generation  of 
algorithms  which  will  evenually  replace  visual  editing  by  an  automated 
editing  process* 

A  database  was  assembled  of  eight  types  of  artifacts  identified  by  visual 
inspection  of  the  EEQ's  strip  chart  for  the  bimanual  study*  The  study 
involved  scalp  recordings  made  from  twenty-six  sites*  and  additional 
channels  for  vertical  and  horizontal  eye  movements  and  for  muscle 
potentials  from  the  flexor  digitorium*  aPz  was  used  as  the  reference 
eleotrode*  A  Bioelectric  Instruments  Model  AS-64P  amplifier  with  *10  to  50 
Hz  passband  (1?  db/ootave  rolloff)  was  used  with  a  full-scale  gain  of  120 
uv*  EEQ's  were  monitored  on  two  e i ght -ohanne I  Beckman  Accutraces  and  on 
one  sixteen-channel  Beckman  Accutrace*  Paper  speed  was  set  at  15  mm/sec* 

The  eight  types  of  artifacts  are i  vertical  eye  movements  (VEO Q) *  horizontal 
eye  movements  (HEOQ) *  muscle  contamination  (EMQ)«  electrode  sways  (SWAY)* 
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876 

769 

787 

1 

61% 

399fal 

1 

33% 

24% 

21% 

22% 

PUL 

1 

4910 

270 

4640  11216 

1225 

1239 

1230 

1 

5% 

95%  1 

25% 

25% 

25% 

25% 

BVC 

1 

2123 

— 

—  1 

593 

501 

504 

525 

1 

1 

28% 

24% 

24% 

25% 

DUL 

1 

3824 

— 

—  1 

956 

956 

956 

956 

1 

1 

1 

25% 

25% 

25% 

25% 

TABLE  3:  Distribution  of  Artifacts  in  Data  Base. 
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groee  movements  < GMO ) ,  electrode  pops  (POP),  electrode  drying  up,  loose  or 
deed  (OUL),  and  pulse  and  EKG  (PUL),  The  criteria  according  to  which  the 
artifacts  were  characterized  will  be  discussed  below.  The  goal  of  this 
study  was  to  provide  a  database  for  the  generation  of  algorithms  which  will 
permit  the  eventual  automation  of  the  current  visual  editing  procedure. 

One  run  from  each  participant  was  randomly  selected  for  analysis.  This 
provided  a  total  of  9  sample  runs,  representative  of  the  beginning,  middle 
end  of  the  5  to  7  hour  recording  sessions.  Each  artifact  was 
character i zed  according  to  where  it  occurred  in  a  given  6  sec  trial  of  the 
run,  i.e.,  during  the  cue  (CUE),  stimulus  (STIM),  movement  (MOVE)  or 
feedback  (FOBK )  events  of  each  trial.  Therefore,  in  any  given  trial,  a 
single  artifact  could  have  occurred  at  four  p i aces ;  since  there  were 
generally  171  trials  per  run,  there  were  684  possible  occurrences  (4  x  171) 
of  each  type  of  artifact.  More  than  one  artifact  could  have  occurred 
simultaneously,  Artifacts  were  also  character i zed  according  to  whether  they 
were  gross  or  moderate  (VEOG,  HEOG,  SWAY,  GMO),  and  regular  or  irregular  in 
the  case  of  the  contami nat i on  by  pulse  (PUL).  EMG  and  DUL  were  not 
evaluated  according  to  degree  of  severity. 

Table  3  shows  the  total  number  of  each  type  of  artifact  that  occurred  over 
all  the  runs  included  in  this  study,  as  well  as  the  proportion  which  were 
identified  as  gross  or  as  moderate.  In  addition,  Table  3  shows  the 
distribution  of  the  artifacts  across  the  four  types  of  events  in  terms  of 
absolute  number  and  in  terms  of  what  proportion  that  number  represents  of 
the  total  number  of  occurrences. 


b.  Criter  La  for  I  den  t.  i  f  i  cation  of  Artifacts. 

VEOGi  Contamination  by  an  eyeblink  was  considered  to  be  gross  VEOG 
artifaot.  Gross  VEOG  artifacts  often  contaminated  the  frontal  (aFI,  aF2, 
F3,  F4 )  brain  channels  and  represented  forty-three  percent  of  the  total 
number  of  VEOG  artifacts.  An  example  of  gross  VEOG  contamination  is  found 
in  Figure  29.  Moderate  contamination  generally  occurred  as  a  result  of 
movement  of  the  VEOG  electrodes  and  represented  fifty-seven  percent  of  the 
total  eye  contamination.  An  example  of  moderate  VEOG  contamination  is 
shown  in  Figure  30. 

HEOGt  Saccades  or  shifts  due  to  horizontal  gaze  were  the  most  common  types 
of  HEOG  artifaot.  An  HEOG  artifact  was  identified  as  being  gross  if  it 
exceeded  approximately  400  msec  in  duration  or  about  50  uv  in  amplitude. 
Overall,  32%  of  the  HEOG  artifacts  were  considered  to  be  gross  and  68% 
moderate.  An  example  of  a  gross  HEOG  artifact  is  shown  in  Figure  31.  An 
example  of  moderate  a  HEOG  artifact  is  shown  in  Figure  32. 

SWAY t  Electrode  sways  appearing  on  the  strip  chart  write-outs  may  be 
largely  attributed  to  movements  of  the  electrode  wire  or  to  movement  of  the 
quiok  insert  ball  in  the  plastic  holder.  Movement  of  the  hat  itself  or 
small  subtle  head  movements  may  also  be  reflected  as  »  sway  of  an 
eleotrode.  A  sway  with  a  peak-to-peak  drift  exceeding  approximately  100  uv 
in  2  sec  was  considered  to  be  a  gross  artifact.  More  moderate  sways  were 
character i zed  with  a  peak-to-peak  drift  of  less  than  approximately  100  uv 
in  2  seo,  or  alternatively  with  an  amplitude  drift  exceeding  approximately 
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100  uv  over  •  longer  period  of  time.  Table  3  shows  that  the  distribution 
of  sways  was  about  constant  across  all  events.  Essentially,  this  means  ti«at 
sways  tended  to  occur  throughout  trials,  and  not  at  brief  intervals  within 
a  trial.  In  addition,  as  shown  in  Table  3,  33%  of  the  sways  were  noted  to 
be  moderate  and  13%  to  be  gross.  An  example  of  a  gross  electrode  sway  is 
shown  in  Figure  33,  An  example  of  a  moderate  sway  is  shown  in  Figure  34, 

ELECTRODE  POPSi  The  degree  of  severity  of  the  eleotrode  pop  was  determined 
aooording  to  its  amplitude.  Those  electrode  pops  with  an  amplitude 
exceeding  about  SO  uv  were  considered  to  be  gross,  with  moderate  electrode 
pops  having  an  amplitude  of  less  than  about  SO  uv.  Table  3  shows  that  81% 
of  the  eleotrode  pops  were  considered  to  be  moderate,  and  13%  gross.  An 
example  of  a  gross  electrode  pop  is  shown  in  Figure  33,  A  moderate 
eleotrode  pop  is  shown  in  Figure  36, 

6R0SS  MOVEMENTS  Large  deviations  (artifacts)  in  the  EEG  tracings  are  often 
the  result  of  head  movement.  In  these  cases,  the  resulting  artifact  was 
identified  as  being  gross,  as  distinguished  from  a  more  moderate  artifact 
due  to  movmmmnt,  when  the  size  of  the  signal  saturated  the  strip  chart  for 
that  channel,  Sixty*one  percent  of  the  movement  artifacts  were  identified 
as  being  gross,  and  33%  were  identified  as  being  moderate,  Examples  of  GMO 
contamination,  gross  and  moderate,  are  shown  in  Figures  37  and  38, 
respect i vel y , 

PULSE/EKGi  The  contamination  by  pulse  or  EKG,  a  transient  periodicity  in 
the  EEG,  was  oharaoter i zed  as  either  regular  or  irregular.  Pulse  was 
character i zed  as  irregular  (33%)  primarily,  and  was  characterized  as  being 
regular  only  a  small  proportion  of  the  time  (3%).  An  example  of  regular 
pulse  is  shown  in  Figures  39,  An  example  of  irregular  pulse  in  the  EEG  is 
shown  in  Figure  40, 

EMQt  Musole  potential  (EM6)  contamination  of  the  EEG  was  also  identified, 
and  was  noted  to  oocur  especially  in  the  frontal  and  temporal  channels.  It 
was  noted  that  EMQ  present  during  the  CUE  event  tended  to  remain  present 
throughout  the  entire  trial.  Examples  of  EMG  are  shown  in  Figure  41  for 
frontal  ohannels  and  Figure  42  for  temporal  ohannels. 

ELECTRODE  DRYING  UP/L00SE/0EA0  i  This  type  of  artifact  was  due  to  poor  (if 
any)  oonduotion  between  the  scalp  and  the  electrode.  The  contaot  may  be 
insufficient,  ( loose i  Figure  43)  as  a  result  of  the  conductive  gel  drying 
up.  A  dead  electrode  (flat  EEG  traoei  Figure  44)  tends  to  indicate  that 
there  is  no  oonduotion  at  all,  It  is  not  surprising,  then,  that  when  the 
artifaot  was  present  in  one  event  of  a  trial,  it  was  present  for  the  entire 
trial.  These  types  of  artifacts  were  not  considered  in  terms  of  degree  of 
sever i ty . 
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